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NOTICE 


When  Government  drawings,  specifications,  or  other  data  are  used  for  any 
purpose  other  than  in  connection  with  a  definitely  Government-related 
procurement,  the  United  States  Government  incurs  no  responsibility  or  any 
obligation  whatsoever.  The  fact  that  the  Government  may  have  formulated  or  in 
any  way  supplied  the  said  drawings,  specifications,  or  other  data,  is  not  to 
be  regarded  by  implication,  or  otherwise  in  any  manner  construed,  as  licensing 
the  holder,  or  any  other  person  or  corporation;  or  as  conveying  any  rights  or 
permission  to  manufacture,  use,  or  sell  any  patented  invention  that  may  in  any 
way  be  related  thereto. 

The  Public  Affairs  Office  has  reviewed  this  paper,  and  it  is  releasable  to  the 
National  Technical  Information  Service,  where  it  will  be  available  to  the 
general  public,  including  foreign  nationals. 

This  paper  has  been  reviewed  and  is  approved  for  publication. 


HENDRICK  W.  RUCK,  Technical  Advisor 
Training  Systems  Division 


RODGER  D.  BALLENTINE,  Colonel,  USAF 
Chief,  Training  Systems  Division 


This  technical  paper  is  printed  as  received  and  has  not  been  edited  by  the 
AFHRL  Technical  Editing  staff.  The  opinions  expressed  herein  represent  those 
of  the  author  and  not  necessarily  those  of  the  United  States  Air  Force. 
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1  EXECUTIVE  SUMMARY 


This  document  contains  a  description  of  a  prototype  Advanced  On-the-job  Training 
System  (AOTS)  that  was  designed  and  developed  by  the  Air  Force  Human  Resources 
Laboratory  (AFHRL).  Further,  it  identifies  the  users  of  the  prototype  AOTS,  the 
functions  they  will  perform,  and  how  the  prototype  AOTS  provides  automated  support  for 
the  performance  of  those  functions.  This  document  was  developed  for  use  primarily  by 
personnel  within  those  operational  workcenters  that  have  been  chosen  to  participate  in  a 
System  Level  Test  and  Evaluation  (SLT&E)  of  the  prototype  AOTS.  The  purpose  of  this 
document  is  to  provide  the  information  needed  to  transition  from  the  conventional 
On-the-Job  Training  (OJT)  system  to  the  prototype  AOTS. 

AFHRL  was  tasked  by  the  Air  Staff  to  design,  develop,  and  test  the  prototype  AOTS 
within  an  operational  setting.  This  initiative  was  established  as  a  result  of  findings 
during  studies  of  on-the-job  training  programs  within  operational  units.  Reports  of  these 
studies  identified  several  deficient  areas  within  the  on-the-iob  training  system.  The 
timeline  for  the  design,  development,  and  test  of  the  prototy  ie  AOTS  is  presented  in 
Figure  1-1. 
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Figure  1-1  The  AOTS  Master  Schedule 


The  prototype  AOTS  is  a  computer-based  training  system  that  employs  modern 
computer  technology  to  develop  and  deliver  instructional  and  test  materials,  manage 
trainee  progress  through  full  duty  position  qualification,  and  evaluate  the  effectiveness 
and  efficiency  of  the  training  system.  The  data  base  for  this  system  consists  of  the 
following  data: 

a)  Automated  Airman  Training  Records  -  personnel  data,  the  status  of  ongoing 
training  and  the  history  of  completed  training; 

b)  Master  Task  Lists  -  the  tasks  assigned  to  each  Air  Force  Specialty,  to  include 
the  task  elements; 

c)  Behavioral  Objectives  -  terminal  and  supporting  behavioral  objectives  to 
support  training  and  evaluation  of  tasks  and  subtasks; 

d)  Evaluation  Materials  •  knowledge  and  performance  tests  for  use  in  determining 
attainment  of  behavioral  objectives; 


P*ge  1 


AOTS  Operational  Guide 


e)  Other  Training  Requirements  -  ancillary  training,  additional  duty  training, 
contingency  tasks,  and  career  development  courses; 

f)  Event  Schedules  -  training  and  evaluation  event  schedules,  and  lists  of 
resources  required  for  each;  and 

g)  System  Evaluation  Data  -  test  results,  training  times,  airmen  qualifications, 
and  other  data  used  to  indicate  training  efficiency  and  effectiveness. 


Bergstrom  AFB,  TX  was  chosen  as  the  development  site  for  the  prototype  AOTS. 
Bergstrom  and  Ellington  ANGB,  TX  have  been  selected  as  the  sites  at  which  the  SLT&E 
will  occur.  All  Air  Force  components;  active,  reserve,  and  guard,  will  participate  in  the 
SLT&E.  The  following  five  AFSs  will  be  included  in  the  test. 

a)  Jet  Engine  Maintenance,  Air  Force  Specialty  Code  (AFSC)  426X2; 

b)  Aircraft  Maintenance,  AFSC  431X1; 

c)  Personnel,  AFSC  732X0; 

d)  Security  Police  -  Security,  AFSC  811X0;  and 

e)  Security  Police  -  Law  Enforcement,  AFSC  811X2. 

Workcenters  within  the  67th  Tactical  Reconnaissance  Wing  (active)  and  the  924th 
Tactical  Fighter  Group  (reserve)  at  Bergstrom,  and  the  147th  Fighter  Interceptor  Group 
(Guard)  at  Ellington,  will  participate  in  the  SLT&E.  All  participants  will  help  determine 
if  the  prototype  AOTS  meets  established  goals,  and  whether  Air  Force-wide  implementa¬ 
tion  should  be  considered. 

Figure  1-2  shows  the  five  major  subsystems  that  constitute  the  prototype  AOTS.  The 
core  of  AOTS  is  the  data  base  itself.  It  is  within  the  data  base  that  information  is  stored, 
retrieved,  user  interacts  with,  and  reported  upon. 


Figure  1-2  The  Advanced  On-the-job  Training  System  -  AOTS 
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The  prototype  AOTS  is  using  a  central  computer  system  with  Z-248  Personal 
Computers  (PCs)  as  local  AOTS  terminals.  Figure  1-3  provides  a  diagram  of  the  system 
configuration.  High  speed  digital  communication  lines  between  the  central  computer  and 
the  Z-248s  are  provided.  More  than  85  PCs  are  used  throughout  the  AOTS.  The  central 
computer  is  capable  of  handling  more  than  10  million  operations  per  second,  and  storing 
more  than  1300  million  characters  of  data.  The  workcenter  users  are  provided  printing 
capabilities,  and  optical  mark  readers  for  scoring  tests  that  are  administered  off-line. 
Appendix  B  provides  details  on  the  implementation  and  installation  of  the  prototype 
AOTS. 


Figure  1-3  The  Prototype  AOTS  Hardware  Configuration 
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2  INTRODUCTION 


2.1  Purpose 


The  purpose  of  this  Operational  Guide  is  to  provide  the  information  needed  during 
the  System  Level  Test  and  Evaluation  (SLT&E)  of  the  prototype  Advanced  On-the-job 
Training  System  (AOTS)  to  transition  from  the  conventional  Air  Force  on-the-job  training 
system  to  the  prototype  AOTS.  This  description  has  been  developed  primarily  for  use  by 
those  personnel  assigned  to  the  operational  workcenters  that  have  been  chosen  to 
participate  in  the  SLT&E  of  the  prototype  AOTS. 


2.2  System  Level  Test  and  Evaluation  (SLT&E)  Period 


The  prototype  AOTS  will  undergo  SLT&E  during  the  period  1  August  1988  through 
31  July  1989.  The  SLT&E  is  intended  to  demonstrate  a  proof-of-concept  (i.e.  to  attempt  to 
prove  that  the  prototype  AOTS  provides  the  functionality  necessary  to  meet  the 
requirements  of  the  Air  Force  on-the-job  training  system,  and  that  the  prototype  AOTS  is 
more  effective  and  efficient  than  the  conventional  on-the-job  training  system).  Data 
required  to  substantiate  that  the  concept  works  will  be  collected  during  the  SLT&E  period 
and  analyzed  and  reported  at  the  end  of  the  period.  All  computer  hardware  and  software 
necessary  to  support  the  prototype  AOTS  will  be  made  available  to  the  participating 
organizations  prior  to  the  start  of  SLT&E.  Also  prior  to  the  start  of  the  SLT&E  personnel 
participating  will  receive  training  on  how  to  operate  the  system  and  on  the  use  of  system 
generated  products. 


2.3  AOTS  Description 


The  prototype  AOTS  is  a  computer-based  training  system  designed  to  provide 
automated  support  for  the  functional  requirements  of  the  Air  Force  on-the-job  training 
system.  The  AOTS  is  comprised  of  five  subsystems.  Each  subsystem  is  made  up  of 
components  that  perform  specific  functions.  Figure  2-1  illustrates  the  functional 
hierarchy  of  the  prototype  AOTS  including  the  five  subsystems  and  the  components  of 
each  subsystem.  The  functions  and  how  they  are  applied  are  described  within  subsequent 
sections  of  this  plan. 

The  Management,  Evaluation,  and  Computer  Assisted  Training  subsystems  are  the 
principal  subsystems  within  the  prototype  AOTS  that  provide  training  support  functions. 
The  Computer  Support  Subsystem  supports  AOTS  users  by  providing  the  necessary 
hardware  and  software  to  interact  with  the  AOTS  data  base.  The  Personnel  and  Support 
subsystem  defines  the  personnel  requirements  for  AOTS,  specifically,  the  maintenance  and 
logistics  requirements,  and  the  human  factors  (or  man/machine  interface)  requirements. 
All  interna!  interfaces  between  subsystems  are  fully  integrated  together. 
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2.4  AOTS  External  Interfaces 


For  an  AOTS  to  function  effectively,  it  must  exchange  data  with  other  Air  Force 
systems.  These  external  interfaces  have  been  established  for  the  prototype  AOTS  and  are 
illustrated  in  Figure  2-2.  A  fully  operational  AOTS  (one  that  has  been  approved  for  Air 
Force-wide  use)  would  interface  electronically  with  external  Air  Force  systems.  However, 
the  interfaces  between  the  prototype  AOTS  and  these  external  systems  will  be 
accomplished  manually. 


Figure  2-2  AOTS  External  Interfaces 
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2.5  AOTS  Requirements/Functions  and  the  Subsystems  Matrix 

The  AOTS  requirements  have  been  matched  up  to  each  of  the  subsystems.  Table  2-1 
provides  an  cross  reference  matrix  of  the  AOTS  requirements  and  functions  to  each  of  the 
five  subsystems  in  AOTS. 


REQUIREMENTS 

PRINCIPAL  FUNCTIONS  PERFORMED  BY  THE  AOTS 

BENEFITS 

MANAGEMENT 

SUBSYSTEM 

EVALUATION 

SUBSYSTEM 

COMPUTER  SUP¬ 
PORT  SUBSYS¬ 
TEM 

mm 

PERSONNEL  AND 
SUPPORT  SUB¬ 

SYSTEM 

OJT  Administra¬ 
tion 

Accurately  Identi¬ 
fies  training  for 
fufl  position  quali¬ 
fication 

Provides  auto¬ 
mated  training 
Identified  support 
keeps  training 
records 

Identifies  training 
requirements  for 
system  managers 
and  users 

Redirects  OJT  from  gener-  1 
el  career  development  to 
specific  position  qualifica¬ 
tion.  Relieves  supervisor 
of  administrative  dor  -ten- 
tatlon  burden. 

OUT  Management 

Manages  learning 
activities  and 
controls 

progress,  sched¬ 
ules  training  and 
manages  re¬ 
source  allocation. 

Provides  auto¬ 
mated  support 
for  prioritizing 
training  require¬ 
ments. 

Efficiently  employs  "big 
picture"  resource  and 
training  program  manage¬ 
ment.  More  accurate  ca¬ 
pacity  estimates. 

Training  Evalua¬ 
tion/Assessment 

Identifies,  selects, 
and  delivers  eval¬ 
uation  Instru¬ 
ments. 

Provides  on-line 
delivery,  support 
off-line  delivery, 
keeps  evaluation 
and  assessment 
records. 

Enables  more  frequent 
task  proficiency  evalua¬ 
tions  and  unit  training 
effectiveness  assessments 
to  ensure  quality  training. 

System  Evalua¬ 
tion 

Performs  QA  and 
System  evalua¬ 
tion. 

Collects,  stores, 
calculates,  and 
delivers  reports. 

Enables  system  quality 
assurance  end  reliability 
checks. 

Data  Automation 
Support 

#«|| 

Provides  user-friendly  sin¬ 
gle-source  support 

Soecification  of 
AOTS  OoeratlonaJ 
Requirements 

Identifies  person¬ 
nel  and  organiza¬ 
tional  requiremen¬ 
ts  for  AOTS. 
Identifies  imple¬ 
mentation,  opera¬ 
tion,  and  expan¬ 
sion  requiremen¬ 
ts. 

Provides  for  overall  transi 
tlon  and  expansion  of  the 
operational  system. 

Development  of 
Evaluation  and 
Training  Materials 

Identifies  and  de¬ 
velops  task-spe¬ 
cific  evaluation 
instruments. 

Supports  devel¬ 
opment  Stores 
materials. 

Develops 
task-specific  eval¬ 
uation  and  train¬ 
ing  materials. 

Provides  needed  training 
matertals/devices  with  effi¬ 
cient  balance  between 
centralization  and  decen¬ 
tralization. 

Delivery  of  Evalu¬ 
ation  and  Training 
Materials 

Delivers  on  and 
off  line  evaluation 
instruments. 

MH 

Provides  training 
on-line  or  prints 
for  off-line  use. 

Employs  new  technology 
and  efficiencies  m  training 
techniques. 

Table  2-1  Requirements/Functions  Cross  Reference  Matrix 
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3  SCOPE 


3.1  Scope  of  the  Operational  Guide 

This  Operational  Guide  explains  how  the  prototype  AOTS  will  be  integrated  into  the 
on-the-job  training  programs  of  those  operational  workcenters  selected  to  participate  in 
the  SLT&E.  It  defines  the  users  of  the  prototype  AOTS.  It  outlines  user  responsibilities 
in  developing,  conducting,  receiving  and  managing  on-the-job  training  within  the 
applicable  functional  areas  and  workcenters;  and  for  operating  and  maintaining  the 
prototype  AOTS.  Further,  it  explains  how  automation  will  support  the  specific  processes 
associated  with  providing  on-the-job  training. 


3.2  Participating  Organizations  and  Air  Force  Specialties  (AFSs) 

Operational  workcenters  from  all  Air  Force  components  (Active,  Guard  and 
Reserve)  will  be  participating  in  the  SLT&E.  Figure  3-1  defines  the  principal  participants 
and  the  development  command  (AFSC).  AOTS,  a  program  sponsored  by  AF/DPP,  was 
given  over  to  AFSC  (in  turn  to  HSD  and  AFHRL)  with  TAC,  AFRES,  and  ANG  being  the 
test  participants.  Active  and  Reserve  organizations  are  located  at  Bergstrom  AFB,  TX 
and  the  Guard  organizations  are  located  at  Ellington  ANGB,  TX.  Additionally,  civilian 
contractors  are  assisting  the  Air  Force.  The  organizations  and  units  that  participate  in 
the  SLT&E  of  the  prototype  AOTS  are  listed  in  Figures  3-2. 


Figure  3-1  The  AOTS  Air  Force  Major  Participants  and  Commands 
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Figure  3-2  The  AOTS  Participants 


3.3  Background 

In  the  Air  Force  environment  enlisted  skills  are  acquired  and  maintained  primarily 
through  on-the-job  training.  The  increasing  complexity  of  weapon  systems  and  equip¬ 
ment,  the  loss  of  qualified  middle  management  personnel,  and  increasing  mission  demands 
have  made  job-site  training  more  difficult  to  conduct.  An  increase  in  attention  to 
training  effectiveness  and  efficiency  has  surfaced  indications  of  serious  problems  within 
the  on-the-job  training  system.  As  an  initiative  to  formally  address  problems,  the  Air 
Staff  requested  that  AFHRL  conduct  a  study  of  the  on-the-job  training  system 
(Stephenson,  R.W.  and  Burkett,  J.R.,  On-the-Job  Training  in  the  Air  Force:  A  Systems 
Analysis.  AFHRL-TR-75-83,  December  1975).  Subsequent  to  the  AFHRL  study  the  Air 
Staff  requested  the  Air  Force  Inspector  General  perform  a  Functional  Management 
Inspection  (FMI)  of  the  on-the-job  training  system  (The  Air  Force  Inspector  General 
Functional  Management  Inspection  (FMI)  ot  On-the-Job  Training  in  the  Air  Force,  PN 
76-269,  April  1977).  Findings  during  the  study  and  the  FMI  indicated  the  need  for 
specific  improvements  within  the  system.  These  findings/recommendations  are  reflected 
in  Figure  3-3  and  along  with  subsequent  actions. 
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Figure  3-3  Inspector  General’s  Findings/Recommendations  and  Subsequent  Actions 


3.3.1  System  Definition  Study 

Subsequent  to  the  AFHRL  study  and  the  FMI  the  Air  Staff  developed  initiatives 
aimed  at  correcting  the  identified  deficiencies.  The  ultimate  initiative  was  to  develop 
and  test  a  prototype  training  system.  The  Air  Staff  specified  that  the  prototype  AOTS 
would  be  developed  and  tested  within  the  actual  operational  environment.  This  initiative 
included  determining  the  feasibility  of  applying  automation  within  the  on-the-job 
training  environment.  The  first  step  toward  developing  a  prototype  training  system  was 
to  conduct  another  study  which  was  to  result  in: 

a)  The  definition  of  the  Air  Force  on-the-job  training  system  functional 
requirements; 

b)  the  determination  of  how  automation  could  be  applied  to  improve  training 
system  effectiveness  and  efficiency; 

c)  a  documented  system  description;  and 
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d)  a  determination  as  to  which  Air  Force  site  was  best  suited  as  the  location  for 
developing  and  testing  the  prototype  AOTS. 


A  contract  was  awarded  to  the  Science  Applications  International,  Corporation  to  assist 
the  AFHRL  in  accomplishing  the  study.  The  results  of  this  study  are  documented  in  an 
AFHRL  Technical  Paper  (AFHRL-TP-83-54),  entitled  "Integrated  Training  System  for  Air 
Force  On-The-Job  Training:  Specification  Development",  dated  March  1984. 


3.3.2  Design  and  Development  of  the  Prototype  AOTS 

In  August  1985  a  three-phase  contract  for  the  design  and  development  of  the 
prototype  AOTS  was  awarded  to  the  Douglas  Aircraft  Company.  The  time  periods  for 
these  phases  can  be  found  in  Figure  1-1.  The  requirements  of  Phase  I  were  to  develop  a 
preliminary  design  for  the  AOTS  and  each  of  its  subsystems,  to  produce  written 
functional  specifications  for  the  AOTS  and  its  subsystems,  and  to  produce  written 
software  specifications  for  each  subsystem.  The  requirements  of  Phase  II  were  to  develop 
the  detailed  design  for  the  subsystems,  to  develop  the  required  software  packages,  to 
acquire  the  computer  hardware  to  be  used  within  the  workcenters  participating  in  the 
SLT&E,  and  to  make  all  preparations  necessary  for  the  SLT&E  to  occur.  Phase  III  is  for 
the  SLT&E  of  the  prototype  AOTS. 
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4  THE  OPERATIONAL  ENVIRONMENT  AND  USERS  OF 
THE  PROTOTYPE  AOTS 


4.1  The  Operational  Environment 


Figure  4-1  lists  the  organizations,  units,  and  workcenters  participating  in  the  SLT&E 
of  the  prototype  AOTS.  The  environment  in  which  the  prototype  AOTS  has  been 
developed,  and  will  be  tested,  was  selected  because  it  is  representative  of  the  typical  Air 
Force  operational  environment.  The  AFSs  selected  (see  Figure  4-1)  represent  the  largest 
and  most  diverse  in  the  Air  Force,  and  encompass  various  types  of  disciplines.  The 
missions  performed  by  these  AFSs  at  the  chosen  locations  are  representative  of  the 
missions  performed  at  most  Air  Force  locations.  Developing  and  testing  the  prototype 
AOTS  in  a  representative  operational  environment  enables  a  valid  assessment  as  to 
whether  or  not  the  system  is  sufficiently  flexible  to  accommodate  the  requirements  within 
all  functional  areas. 


4.2  Users  of  the  Prototype  AOTS 


Figure  4-1  reflects  the  types  of  users  of  the  AOTS  and  the  development/support 
personnel.  This  section  of  the  Operational  Guide  defines  the  functions  to  be  performed 
by  these  users  and  the  types  of  interactions  with  the  prototype  AOTS  when  the  functions 
arc  supported  by  automation.  The  functions  performed  within  the  conventional 
on-the-job  training  system  will  continue  to  be  performed  within  the  prototype  AOTS. 
However,  the  methods  for  accomplishing  these  functions  have  been  greatly  simplified. 
Additionally,  new  functions  are  required,  which  serve  to  improve  the  quality  of  training. 


Figure  4-1  The  Operational  Users 
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4.2.1  Trainee 


This  type  of  user  is  the  targeted  recipient  of  training  provided  within  operational 
workcenters.  Each  enlisted  person  assigned  to  the  workcenters  participating  in  the  SLT&E 
of  the  prototype  AOTS  is  a  potential  trainee.  Only  those  persons  who  are  fully  position 
qualified  and  who  do  not  require  other  training  during  the  entire  period  of  the  SLT&E 
will  be  exempted  from  this  category.  Where  trainees  are  currently  required  to  research 
manual  forms  to  determine  their  training  requirements  and  progress  status  the  AOTS  will 
provide,  on  line,  a  complete  history  of  those  AFS  tasks  which  have  been  certified  and 
other  training  accomplishments,  and  the  status  of  all  training  underway.  Within  the 
AOTS  the  status  of  training  progress  will  include  event  level  data  (i.e.  knowledge  training, 
knowledge  evaluation,  performance  training,  performance  evaluation)  for  AFS  tasks. 
Additionally,  trainees  will  be  able  to  identify  and  access  materials  that  are  available  for 
knowledge  training  and  knowledge  testing.  These  capabilities  will  enable  trainees  to  take 
advantage  of  unscheduled  training  opportunities. 


4.2.2  Trainer 


This  type  user  interacts  with  a  trainee  to  teach  him/her  the  knowledge,  skills,  and 
performance  associated  with  specified  tasks.  When  a  trainee  is  assigned  tasks  for  training 
the  workcenter  supervisor  selects  from  within  the  same  workcenter  the  person  who  is  best 
qualified  to  teach  the  applicable  tasks,  and  designates  that  person  as  the  trainer.  When 
selected  as  a  trainer  a  person  is  provided  capabilities  to: 

a)  review  the  on-line  records  of  the  trainee  to  determine  the  training 

requirements  and  the  status  of  progress; 

b)  review  the  task  records  to  determine  the  subtasks,  activities,  supporting 

knowledge  and  skills  and  other  task  elements,  the  sequences  in  which  training 
must  be  accomplished,  and  the  resources  required  for  task  performance; 

c)  review  the  behavioral  objectives  written  for  the  tasks  to  determine  the 

required  outcomes,  the  materials  available  to  support  knowledge  training,  and 
resources  required  for  training;  and 

d)  schedule  training  events. 


4.2.3  Evaluator 

This  type  user  interacts  with  a  trainee  to  either  administer  a  written  knowledge  test 
or  to  observe  and  rate  trainee  performance  of  a  task.  The  process  for  selecting  an 
evaluator  is  the  same  as  that  for  a  trainer.  An  evaluator  is  given  access  to  the  same 
system  capabilities  as  a  trainer,  plus  an  evaluator  is  given  access  to  evaluation  materials 
to  be  used  off  line. 


4.2.4  Supervisor 

This  type  user  is  the  individual  within  an  operational  workcenter  who  is  responsible 
for  ensuring  that  training  programs  are  properly  planned  and  executed,  and  records  of 
trainee  progress  and  qualifications  are  accurately  maintained.  This  user  is  provided  with 
capabilities  to: 

a)  review  training  records  of  all  personnel  supervised; 
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b)  develop  lists  of  position  requirements; 

c)  prioritize  training  requirements; 

d)  schedule  training  and  evaluation  events; 

e)  review  task  records; 

f)  review  behavioral  objectives; 

g)  obtain  test  materials  for  off-line  use; 

h)  certify  trainee  qualifications;  and 

i)  request  trainee  progress  and  system  effectiveness  reports. 


4.2.5  Training  Manager 


This  user  is  responsible  for  assisting  supervisors  in  the  development  of  effective 
training  programs  and  providing  overall  management  for  training  programs  at  the  unit 
level  (Squadron,  Detachment)  or  higher  levels  (Deputy  Commander  for  Maintenance 
(DCM)  Training  Management,  Consolidated  Base  Personnel  Office  (CBPO)  On-the-Job 
Training  Management).  This  user  is  provided  with  capabilities  to: 

a)  review  the  training  records  of  all  personnel  assigned  to  the  organizations  for 
which  he/she  is  responsible; 


b)  review  all  training  requirements  for  these  personnel; 

c)  schedule  training  and  evaluations  for  these  personnel; 

d)  obtain  test  materials  for  off-line  use;  and 

e)  request  trainee  progress  reports  and  training  effectiveness  reports. 


4.2.6  Quality  Control  Administrator 


This  user  (who  may  also  be  a  Training  Manager)  will  perform  all  functions 
associated  with  administering  Quality  Control  Evaluations.  He/she  will  assess  the 
effectiveness  of  the  automatic  on-line  process  within  the  prototype  AOTS  that  selects  and 
schedules  Quality  Control  evaluation  events,  and  initiate  changes  as  required.  Hc/shc  will 
also  schedule  events  that  must  be  generated  based  on  off-line  requests.  To  schedule  a 
requested  event  requires  the  Quality  Control  Administrator  to  interact  with  the  prototype 
AOTS  to: 


a)  select  the  task  to  be  evaluated; 

b)  select  the  person  to  be  evaluated; 

c)  select  the  primary  and  alternate  evaluators;  and 
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d)  specify  when  the  event  will  occur. 


4.2.7  Commander 

This  type  user  is  not  envisioned  to  interact  directly  with  the  prototype  AOTS. 
However,  commanders  will  be  provided  reports  generated  from  the  system  for  use  as 
management  tools  in  ensuring  effective  training  programs  are  established  within  all 
applicable  workcenters. 


4.2.8  AOTS  Support  Personnel 

There  is  a  class  of  users  that  support  the  operation  of  AOTS  and  have  developed  the 
software,  and  training  and  evaluation  materials.  All  of  these  users  are  from  AFHRL. 
These  users  are  defined  in  this  section. 


.2.8.1  System  Administrator 

This  type  of  user  has  responsibility  for  general  control  and  administration  of 
unctions  within  the  prototype  AOTS.  This  includes  establishment  of  final  Master  Task 
uists  (MTLs),  ensuring  peronnel  added  to  the  system  are  within  the  approved  workcenters 
and  AFSs,  assignment  of  personnel  to  a  supervisor,  and  other  general  functions. 


4 .2. 8.2  Developer 

This  type  of  user  has  responsibilities  for  analyzing  performance  requirements, 
writing  terminal  and  supporting  behavioral  objectives,  writing  symbolic  test  items,  writing 
Oral  Test  Guides  (OTGs),  writing  Performance  Evaluation  Checklists  (PECs),  developing 
knowledge  and  performance  tests,  and  developing  computer-assisted  instruction.  The 
system  will  provide  this  user  with  capabilities  to: 

a)  enter  the  results  of  task  analysis  on  line; 

b)  author  and  store  behavioral  objectives  on  line; 

c)  author  and  store  test  items  on-line; 

d)  create  tests  on  line  for  both  on-line  and  off-line  use;  and 

e)  author  computer-assisted  instruction  on  line. 


4.2.8.3  System  Programmer 

This  type  of  user  is  responsible  for  the  software  maintenance  for  the  prototype 
AOTS  and  in  this  role  will  have  a  very  broad  access  level  to  allow  problem  identification, 
isolation,  and  correction. 
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5  DEVELOPMENT  OF  TRAINING  MATERIALS  FOR 
THE  PROTOTYPE  AOTS 


Subject  Matter  Experts  (SMEs)  assigned  to  the  AFHRL  have  been  given 
responsibilities  for  the  development  of  all  training  materials  required  during  the  SLT&E 
of  the  prototype  AOTS.  These  responsibilities  and  functions  are  shown  in  Figure  5-1 
under  the  Development  Team  box.  Measures  have  been  taken  to  ensure  that  the  principles 
of  Instructional  Systems  Development  (ISD),  as  defined  in  applicable  Air  Force  directives, 
are  applied  during  all  development  efforts. 


Figure  5-1  Development  Functional  Sequence 


5.1  Analysis  of  Performance  Requirements 

The  first  accomplishment  during  development  was  the  analysis  of  performance 
requirements.  This  analysis  occurred  off  line,  and  included  the  definition  of  the  task 
hierarchy  (see  Table  5-1),  the  preliminary  development  of  task  lists,  the  coordination  of 
these  task  lists  with  the  participating  workcenters  to  all  ensure  requirements  were 
identified,  and  the  breaking  down  of  tasks  into  task  elements.  The  results  were  entered 
into  the  AOTS  MTL  task  records  using  the  MTL  Editor.  This  editor  enables  the  storage, 
within  each  task  record,  of  those  task  elements  reflected  in  Figure  5-2. 
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Analysis  of  Performance  Requirements 


*  Task  Idtntificatioa 

■  Task  Stat«m«nt 

*  Subtasks 

■  Activities 

a  Supporting  Knowlsdgs  and  Skills 

■  Occupational  Survsy  Task  Cross-refsrsncs 

■  References  Governing  Task 

■  Weapon  Systems/Cquipment  Task  Supports 

■  Task  Cross-reference  to  Other  Systems 

■  Locations.  Major  Commands.  Units  and  Workcenters 

■  Determine  Whether  the  Task  must  be  Certified  Prior  to 
being  Performed.  Whether  Task  can  be  Performed  Under 
Supervision,  or  Whether  Task  can  be  Performed  without 
Certification  or  Supervision 

a  Determine  Whether  or  Not  the  Task  Requires  Periodic 
Recertification.  Frequency  of  Certification  (if  required) 

■  Develop  Generic  Position  Requirements  List 

*  Task  Factor  Data 

a  Required  Performance  Resources 
a  Identification  of  Prerequisite  Tasks 

■  Individual  or  Team  Task 

a  Average  Time  to  Qualify  in  a  Task 


AOTS 


r  \ 

AOTS 

DATA  BASE 


Figure  5-2  Analysis  of  Performance  Requirements 


FUNCTION 

DESCRIPTION 

JOB 

The  collective  duties  and  tasks  performed  by  a  person. 

DUTY 

A  segment  of  a  job  that  is  made  up  of  similar  tasks. 

TASK 

A  part  of  a  duty,  a  specific  action,  with  an  objective  and  an  action 
verb,  having  a  definite  beginning  and  an  end.  resulting  in  something 
being  accomplished:  an  observable  and  ratable  action  independent  of 
other  actions  under  the  duty. 

SUBTASK 

A  major  part  of  the  task;  has  the  same  characteristics  as  a  task,  but 
depends  on  other  subtasks  for  task  completion. 

ACTtvrrY 

A  step  within  a  task  or  subtask  that  must  be  accomplished  in  order 
to  perform  the  task  or  subtask;  THE  LOWEST  LEVEL  OF  1 
PERFORMANCE  ACTION  ASSOCIATED  WITR  A  TASK.  I 

SUPPORTING  SKILL/ 
KNOWLEDGE 

A  physical  or  mental  capability  required  to  perform  a  task  or  subtask.  | 

PREREQUISITE  TASK 

A  task  that  must  be  completed  prior  to.  or  during  performance  of.  the  i 
current  task. 

Table  4-1  Performance  Hierarchies 
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5.2  Developing  Behavioral  Objectives 

The  prototype  AOTS  provides  an  editor  for  use  in  authoring  and  storing  Terminal 
and  Supporting  Behavioral  Objectives  (BOs).  Terminal  BOs  are  developed  at  the  task  level 
and  Supporting  BOs  are  developed  at  the  subtask  level.  This  editor  provides  both  a 
novice  and  an  expert  authoring  mode.  The  novice  mode  prompts  the  user  to  author  the 
three  components  (behavior,  conditions,  standard)  of  a  BO  independently,  then  create  the 
narrative  BO  statement  by  combining  the  components.  The  expert  mode  permits  the  user 
to  bypass  the  independent  authoring  of  BO  components  and  directly  author  a  complete 
narrative  BO  statement.  This  editor  also  permits  the  user  to  identify  the  resources 
required  for  training  and  evaluating  the  BO,  and  the  materials  available  for  providing 
requisite  knowledge  training.  The  BO  elements  stored  via  the  BO  Editor  are  reflected  in 
Figure  5-3. 


Figure  5-3  Development  of  Behavioral  Objectives  and  Evaluation  Materials 


5.3  Developing  Evaluation  Materials 

The  next  step  following  the  authoring  of  BOs  was  the  authoring  of  evaluation 
materials.  First,  the  evaluation  strategies  to  be  used  for  each  BO  had  to  be  determined. 
This  determination  was  made  off  line.  The  evaluation  strategies  considered  included: 

a)  over-the-shoulder  observation; 
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b)  product  assessment;  and 

c)  symbolic  testing. 

To  support  the  authoring  of  evaluation  materials  the  AOTS  provides  a  Test  Item  Bank 
(TIB)  Editor.  This  editor  permits  the  authoring  and/or  storing  of  the  elements  shown  in 
Figure  5-3. 


5.4  Developing  Tests 


Following  the  authoring  of  test  items,  the  SMEs  formulated  tests  for  each  BO.  The 
AOTS  provides  a  Test  Editor  for  use  in  formulating  tests.  When  a  performance  test  is 
being  developed,  the  test  editor  allows  one  or  more  OTGs  and  PECs,  developed  for  those 
BOs  applicable  to  the  task,  to  be  included.  This  capability  enables  the  testing  of  one  or 
more  BOs  for  a  task,  using  a  single  test.  When  developing  a  test  to  be  used  for  measuring 
requisite  knowledge  the  test  editor  enables  the  user  to  develop  instructions  for  both 
on-line  and  Cff-line  use  of  the  test.  Also,  test  items  developed  for  one  or  more  of  the  BOs 
applicable  tc  the  task  can  be  designated  for  inclusion  in  the  test.  Again,  this  capability 
permits  the  testing  of  one  or  more  BOs  for  a  task,  using  a  single  test. 


5.5  Developing  Instruction 


SMEs  from  each  AFS  team  were  designated  to  author  Computer  Assisted  Training  to 
support  the  training  of  requisite  knowledge  requirements  for  selected  tasks  (Figure  5-4). 
The  process  developed,  and  employed,  for  determin:  .6  «..te  tasks  for  which  CAI  would  be 
developed  included: 

a)  Establishing  criteria  against  which  the  tasks  would  be  rated 

b)  Assigning  weights  to  the  criteria 

c)  SMEs  rating  each  task  based  on  how  each  criterion  was  met 

d)  Multiplying  each  rating  by  the  weight  assigned  ;o  the  criterion 

e)  Summing  the  products  for  each  task  and  rank-ordering  the  tasks  based  on 
their  total  scores. 


CAI  was  developed  for  as  many  tasks  as  time  permitted  prior  to  the  start  of  SLT&E. 
SMEs  developed  instruction  on  line  using  the  Air  Force  owned  Instructional  Support 
System  (ISS).  Interfaces  between  the  AOTS  and  the  ISS  have  been  developed  to  enable  the 
on-line  delivery  of  instruction  to  trainees. 
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Figure  5-4  Computer  Assisted  Training  Development  Sequence 


5.6  Developing  Generic  Position  Task  Requirments  (GPTR)  Lists 

The  final  requirement  in  the  development  effort  was  to  generate  a  GPTR  list  for 
each  type  of  duty  position  to  be  included  in  the  SLT&E  of  the  prototype  AOTS.  The 
process  for  generating  a  GPTR  is  presented  in  Figure  5-5.  A  GPTR  list  is  comprised  of 
those  tasks  that  are  normally  performed  by  airmen  assigned  to  a  specific  type  of  duty 
position.  The  purpose  for  a  GPTR  is  to  provide  the  baseline  for  the  development  of  an 
Operational  Position  Task  Requirement  (OPTR)  list.  An  OPTR  list  contains  the  specific 
tasks  and  other  training  requirements  associated  with  a  specific  duty  position,  and  is 
tailored  to  the  airman  assigned  to  that  duty  position.  The  prototype  AOTS  provides  a 
capability  to  merge  a  GPTR  list  into  an  OPTR  list,  which  negates  the  need  for  immediate 
supervisors  to  specify  for  inclusion  in  the  OPTR  list  those  tasks  that  are  already 
contained  on  the  GPTR  list.  A  detailed  description  of  how  an  OPTR  is  generated  is 
contained  in  Section  6. 


Figure  5-5  GPTR  Development 
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6  USING  THE  PROTOTYPE  AOTS  IN  THE  OPERATIONAL  ENVIRONMENT 


The  following  describes  how  the  prototype  AOTS  supports  the  training  functions 
performed  within  an  operational  unit.  The  description  starts  with  a  trainee  being 
assigned  to  a  duty  position  within  a  workcenter,  proceeds  through  the  planning  and 
execution  of  a  training  program  for  that  airman,  and  ends  with  the  trainee  becoming  duty 
position  qualified.  Included  are  explanations  of  how  users  interact  with  the  prototype 
AOTS.  Figure  6-1,  The  Airman  Training  Record  Categories  and  Figure  6-2,  Training 
Process  are  provided  to  help  illustrate  training  actions  that  occur  within  an  operational 
unit. 


6.1  Assignment  of  a  Trainee  to  a  Workcenter 

One  of  the  first  actions  to  be  taken  when  an  airman  is  assigned  to  a  workcenter 
designated  to  participate  in  the  SLT&E  of  the  prototype  AOTS  is  to  create  an  automated 
Airman  Training  Record  (ATR)  for  that  airman.  An  ATR  will  normally  be  created  by 
the  unit  training  manager  through  use  of  the  ATR  Editor.  There  will  be  an  exception  for 
personnel  assigned  to  the  participating  workcenters  prior  to  the  start  of  SLT&E  in  that 
their  ATRs  will  be  created  by  the  AFHRL  SMEs.  This  action  is  necessary  to  ensure  the 
availability  of  their  ATRs  on  the  first  day  of  the  SLT&E  period.  The  ATR  contains  nine 
separate  categories  of  data.  These  categories  are  listed  in  Figure  6-1.  Appendix  A 
contains  computer  screen  representations  of  the  ATR.  Review  of  this  material  is  helpful 
in  understanding  how  various  users  interface  with  the  system. 


Figure  6-1  The  Airman  Training  Record  Categories 
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The  actions  assigning  an  airman  to  a  workcenter  include  reporting  the  airman  as 
being  assigned  to  a  specific  duty  position.  This  action  is  coordinated  with  the  unit 
orderly  room.  Duty  positions  are  identified  by  duty  position  numbers  which  are  reflected 
on  Unit  Manning  Documents  (UMDs).  The  duty  position  number  is  significant  to  the  next 
action,  which  is  to  identify  the  tasks  to  be  performed  and  courses  to  be  completed  by  the 
airman  while  assigned  to  that  duty  position.  The  immediate  supervisor  of  the  airman 
identifies  the  tasks  to  be  performed  and  the  courses  to  be  completed  by  the  airman  either 
through  selecting  an  existing  Operational  Position  Task  Requirements  (OPTR)  List  for  the 
duty  position  and  modifying  the  list,  if  deemed  necessary,  or  through  creating  a  new 
OPTR  List  (see  Figure  6-2).  The  identification  number  of  the  OPTR  List  will  be  the  same 
as  the  applicable  duty  position  number  contained  on  the  UMD.  By  using  the  duty  position 
number  as  the  OPTR  List  identification  number,  cross  referencing  the  two  entities  is 
simplified.  It  is  often  necessary  to  assign  more  than  one  airman  to  the  same  duty  position 
number.  If  the  performance  requirements  for  those  airman  are  different  it  becomes 
necessary  to  create  more  than  one  OPTR  L.  st  for  that  duty  position.  Space  is  provided  at 
the  end  of  the  OPTR  identification  .’um^r  where  a  numerical  code  can  be  entered  to 
distinguish  between  OPTR  Lists  for  the  same  duty  position.  An  OPTR  List  is  created 
through  the  OPTR  Editor. 


Figure  6-2  Training  Process 


The  first  step  in  creating  an  OPTR  List  is  to  select  a  Generic  Position  Task 
Requirements  (GPTR)  List  for  the  type  of  duty  position  to  which  the  airman  has  been 
assigned.  (A  GPTR  List  identifies  the  tasks  performed  by  all  persons  assigned  to  the  same 
type  of  duty  position,  e.g.  the  tasks  performed  by  all  Aircraft  Crew  Chiefs  within  Air 
Force  Specialty  Code  (AFSC)  431X1.  GPTRs  are  developed  by  the  AFHRL  SMEs.) 
Subsequent  to  selecting  the  appropriate  GPTR  List  the  immediate  supervisor  commands 
the  AOTS  to  copy  that  GPTR  List  to  the  OPTR  i-ist.  The  GPTR  List  then  becomes  the 
baseline  for  the  OPTR  List.  The  immediate  supervisor  may  then  delete  those  tasks  from 
the  OPTR  List  that  will  not  be  required  for  the  airman. 


The  next  step  is  for  the  immediate  supervisor  to  review  the  MTL  for  the  applicable 
*  AFS  (if  deemed  necessary)  to  determine  if  tasks  exist  that  should  be  added  to  the  OPTR 
List.  If  it  is  determined  that  tasks  on  the  MTL  are  required  then  the  immediate 
supervisor  commands  the  AOTS  to  copy  the  tasks  from  the  MTL  to  the  OPTR  List.  If  an 
immediate  supervisor  discovers  that  the  MTL  does  not  contain  an  AFS  task  that  is 
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required  to  be  performed  by  the  airman  the  capability  to  enter  the  statement  for  that  task 
into  a  temporary  AOTS  file  will  be  provided.  The  requirement  will  be  flagged  to  the 
AFHRL  SMEs  who  will  take  appropriate  development  actions. 

The  next  step  is  to  add  other  tasks  or  courses  to  the  OPTR  List  that  have  been 
designated  as  required  for  all  airmen  assigned  to  the  specific  duty  position.  The 
categories  of  other  possible  requirements  include  Ancillary  Training  courses.  Additional 
Duty  Training  courses,  and  Contingency  Tasks.  Requirements  within  these  categories  that 
have  been  specified  for  the  duty  position  will  have  already  been  identified  within  the 
AOTS  data  base  by  AFHRL  through  use  of  the  Other  Training  Requirements  (OTR) 
Editor.  Any  requirements  within  these  categories  that  have  been  specified  for  the  duty 
position  will  automatically  be  moved  to  the  applicable  OPTR  List  upon  command. 

Subsequent  to  completing  the  OPTR  List  the  immediate  supervisor  will  command  the 
AOTS  to  perform  an  automatic  training  diagnosis  process.  This  process  compares  the 
OPTR  List  to  the  airman’s  ATR  to  determine  if  the  airman  requires  training  in  any  of 
the  tasks  and  courses.  If  any  training  is  required  the  AOTS  produces  an  Individual 
Training  Requirements  (ITR)  List  reflecting  each  requirement.  For  each  AFS  task 
reflected  the  AOTS  assu  ties  that  four  events  are  required: 

a)  Knowledge  Training; 

b)  Knowledge  Evaluation; 

c)  Performance  Training;  and 

d)  Performance  Evaluation. 

For  Ancillary  Training  courses.  Additional  Duty  Training  courses,  and  Contingency 
Tasks  the  AOTS  assumes  that  only  one  training  event  is  required  for  each.  For  Extension 
Course  Institute  and  Career  Development  courses  the  training  manager  or  immediate 
supervisor  may  establish  an  event  for  each  volume  within  a  course  and  up  to  two  events 
for  the  course  examination  and  a  possible  retake. 

Following  the  generation  of  the  ITR  List  the  immediate  supervisor  may  use  the  ITR 
Editor  to  add  other  requirements  to  the  list  which  are  to  be  completed  by  the  airman. 
Other  requirements  in  the  categories  of  Ancillary  Training,  Additional  Duty  Training, 
Contingency  Tasks,  and  Extension  Course  Institute  and  Career  Development  Courses  will 
be  checked  for  validity  within  the  AOTS  whenever  attempts  to  add  them  to  an  ITR  are 
made  (these  requirements  must  have  been  defined  within  the  AOTS  data  base  as  Air 
Force  approved  requirements).  Requirements  within  the  Formal  Training  and  Profession¬ 
al  Military  Education  categories  will  not  be  checked  for  validity  by  the  AOTS  when 
added  to  an  ITR.  (To  acquire  the  computer  space  that  would  be  necessary  to  store  data 
for  the  volume  of  courses  within  these  categories,  and  to  add  the  manpower  that  would  be 
necessary  to  maintain  course  data  currency,  would  not  be  cost  effective  for  the  prototype 
AOTS.)  Since  the  system  does  not  check  for  validity  of  the  requirements  within  these 
categories  immediate  supervisors  will  not  be  restricted  by  the  types  of  requirements  that 
can  be  entered.  This  should  be  beneficial  when  identifying  and  managing  requirements 
such  as  training  provided  at  manufacturers  facilities,  other  formal  training,  or  unique 
local  training.  Following  the  identification  of  requirements  the  immediate  supervisor  will 
identify  the  person(s)  appointed  to  train  and  evaluate  the  airman  on  AFS  tasks.  This  will 
be  accomplished  through  entering  the  social  security  number(s)  of  the  designated  person(s) 
into  the  space  provided. 

The  prototype  AOTS  features  a  qualification  assessment  program.  The  purpose  uf 
this  program  is  to  assist  workcenter  supervisors  in  selecting  airmen  to  be  assigned  tc  ,’uiy 
positions.  This  program  compares  an  airman’s  qualifications  to  duty  position  requiremen¬ 
ts.  Through  this  capability  immediate  supervisors  are  able  to  determine: 
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a)  An  airman’s  qualifications  for  a  single  duty  position 

b)  An  airman’s  qualifications  for  multiple  duty  positions 

c)  The  qualifications  of  multiple  airmen  for  a  single  duty  position 

d)  The  qualifications  of  multiple  airmen  for  multiple  duty  positions. 


The  prototype  AOTS  will  identify,  and  list,  those  requirements  which  would  have  to 
be  trained  should  airmen  be  assigned  to  specified  duty  position(s).  This  feature  will 
enable  immediate  supervisors  to  assign  airmen  to  duty  positions  based  on  who  is  most 
qualified,  and  who  would  require  the  least  amount  of  training  to  become  position 
qualified. 


6.2  Planning  Training 

After  the  immediate  supervisor  has  identified  on  an  ITR  List  all  training 
requirements  for  an  airman  a  training  plan  must  be  developed.  Developing  a  training 
plan  means  to  determine  the  sequence  ir  which  requirements  will  be  trained  within  each 
category.  There  are  two  methods  by  which  AFS  task  training  requirements  on  an  ITR 
List  can  be  sequenced.  First,  the  tasks  are  rank  ordered  in  the  same  sequence  as  that  on 
the  OPTR.  This  process  is  automatic  in  that  the  rank  order  is  carried  forward  from  the 
OPTR.  The  second  method  is  for  the  immediate  supervisor  to  sequence  the  training 
requirements  through  interfacing  with  the  ITR  Editor.  When  tasks  on  an  ITR  are  rank 
ordered  using  the  second  method  the  automatic  rank  ordering  capability  is  lost  for  that 
ITR.  The  second  method  will  be  most  advantageous  when  immediate  supervisors  desire  to 
sequence  training  requirements  to  correspond  with  known  training  opportunities  or  to 
meet  with  established  schedules  for  on-going  training.  Training  requirements  within  all 
categories  other  than  AFS  Tasks  will  be  sequenced  by  the  immediate  supervisor. 


6.3  Scheduling  Training  Events 


After  identifying  training  requirements,  training  events  are  scheduled  using  the 
AOTS  Scheduler.  Training  events  may  be  scheduled  by  the  immediate  supervisor  or  by 
the  training  manager.  Schedules  may  be  established  for  an  individual  airman  or  for 
groups  of  airmen. 

The  first  step  in  establishing  an  event  schedule  is  to  select  either  the  option  to 
schedule  an  individual  airman  or  the  option  to  schedule  a  group  of  airmen.  If  a  schedule 
for  an  individual  airman  is  to  be  established,  the  requestor  may  either  enter  the  social 
security  number  of  the  airman  for  which  the  event  is  being  scheduled  or  command  the 
AOTS  to  display  a  list  of  the  social  security  numbers  for  those  airmea  whose  records  the 
requestor  may  access.  If  a  list  of  the  social  security  numbers  is  displayed  the  requestor 
will  then  select  the  social  security  number  of  the  airman  to  be  scheduled.  Next,  the 
requestor  must  select  the  category  of  the  requirement  to  be  scheduled.  The  requirements 
within  that  category  that  have  been  entered  onto  the  ITR  List  for  the  airman  will  be 
displayed.  If  the  requirement  being  scheduled  is  within  the  AFS  Task  category  the 
requestor  will  then  be  prompted  to  enter  the  date  the  training  event  is  to  occur,  and  the 
identification  of  the  trainer  for  the  requirement  if  different  from  the  trainer  identified 
on  the  ITR.  Otherwise,  the  next  step  is  to  select  the  specific  requirement  to  be  scheduled. 
After  selecting  the  requirement  to  be  scheduled  the  requestor  will  be  prompted  to  enter 
the  dates  and  times  that  the  event  will  start  and  finish,  the  agency  conducting  the 
training,  and  where  the  event  will  occur.  Space  is  provided  for  entering  remarks,  if 
desired.  Once  these  data  are  entered,  an  automatic  search  will  be  performed  to  determine 
if  the  airman  has  already  been  scheduled  for  an  event  during  the  same  time  frame.  If  the 
airman  has  been  scheduled  for  another  event  during  the  same  time  frame,  the  requestor 
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will  be  so  advised.  The  requestor  must  then  decide  whether  to  proceed  with  the  schedule, 
or  to  schedule  the  event  for  a  different  time.  If  the  requestor  proceeds  with  the  schedule 
and  a  scheduling  conflict  is  created,  then  a  determination  must  be  made  as  to  whether 
both  events  can  occur  within  the  same  time  frame  or  if  the  previously  scheduled  event 
should  be  rescheduled. 

To  schedule  a  training  event  for  a  group  of  airmen  the  requestor  will  first  select  the 
category  of  the  requirement.  Next,  the  requestor  will  enter  the  identification  number  for 
the  specific  requirement  to  be  scheduled.  The  requestor  will  then  indicate  the  quota  for 
the  event.  The  AOTS  will  then  automatically  identify,  from  those  airmen  whose  records 
may  be  accessed  by  the  requestor,  those  airmen  who  require  the  training.  The  AOTS  will 
also  identify  the  appropriate  number  of  airmen  who  are  most  eligible  to  receive  the 
training.  Eligibility  will  be  based  on  the  relative  position  of  the  requirement  on  the 
airmen’s  ITRs.  The  requestor  may  either  accept  the  AOTS  produced  listing  of  most 
eligible  airmen  to  be  scheduled  for  the  event  or  interface  with  the  AOTS  Scheduler  to 
substitute  names  of  other  airmen  who  require  the  training.  Airmen  who  do  not  have  the 
requirement  included  on  their  ITR  may  not  be  scheduled  for  an  event.  The  requestor  will 
be  prompted  to  enter  the  dates,  times,  location,  training  agency,  and  remarks  for  the  event 
in  the  same  manner  as  when  scheduling  an  event  for  an  individual  airman.  When  these 
data  are  entered  into  the  AOTS  the  requestor  will  be  advised  of  any  conflicts  in 
scheduling  for  the  affected  airmen.  If  conflicts  are  identified,  the  requestor  must  decide 
whether  to  allow  the  requested  event  to  be  scheduled  within  the  sawte  time  frame  as 
other  scheduled  events  or  to  reschedule  the  event  for  all  airmen  who  rave  a  scheduling 
conflict. 


Training  event  schedules  will  be  printed  and  distributed  to  those  personnel  affected 
by  the  event.  A  schedule  will  reflect: 

a)  the  event  sequence  number; 

b  the  requirement  identification  number; 

c)  the  title  of  the  requirement;  the  names  of  the  personnel  to  receive  training; 

d)  the  dates,  times,  and  location  of  the  event; 

e)  the  agency  conducting  the  training; 

f)  any  resources  required  to  support  the  event;  and 

g)  any  required  instructions. 

Immediate  supervisors  and  training  managers  are  also  provided  the  capability  to 
cancel  scheduled  events. 


6.4  Conducting  Training 

After  training  event  schedules  have  been  established  and  distributed  the  next  thing 
to  occur  is  the  actual  conduct  of  training.  For  the  most  part,  job-site  training  under  the 
purview  of  the  prototype  AOTS  will  be  delivered  in  the  same  way  that  it  is  in  the  current 
system.  Immediate  supervisors  or  other  designated  personnel  will  be  responsible  for 
teaching  airmen  the  required  knowledge  and  skills  required  to  achieve  proficiency  in  AFS 
tasks.  Ancillary  Training,  Additional  Duty  Training,  and  Contingency  Tasks  will  be 
taught  by  agencies  outside  of  the  work  areas.  Completing  required  Career  Development 
Courses  will  be  the  responsibility  of  the  individual  airman.  However,  the  prototype 
AOTS  supports  the  conduct  of  training  for  AFS  tasks  through  providing  data  resulting 
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from  task  analysis  that  arc  not  readily  available  in  the  current  system.  Included  within 
each  task  record  are  the  steps  to  be  accomplished  within  the  task.  These  steps  appear  in 
the  sequence  in  which  they  are  performed.  These  data  serve  as  a  training  outline.  In 
addition,  the  identification  of  publications  that  contain  information  about  the  task,  the 
resources  required  to  perform  the  task,  the  knowledge  and  skills  required  for  task 
performance,  and  the  identification  of  prerequisite  tasks  are  listed.  Further,  the 
behavioral  objectives  that  have  been  developed  for  the  task  are  stored  within  the  task 
record  as  well  as  a  list  of  resources  required  for  training  each  objective,  and  a  list  of 
materials  available  for  providing  knowledge  training  for  each  objective.  These  may 
either  be  displayed  on  line  or  printed  for  off-line  use. 

There  is  an  exception  to  training  under  the  purview  of  the  prototype  AOTS  being 
delivered  in  the  same  way  as  in  the  current  system:  the  delivery  of  knowledge  training 
for  specific  tasks.  The  preceding  paragraph  indicated  that  materials  available  for 
providing  knowledge  training  for  a  behavioral  objective  are  listed  with  that  objective. 
These  materials  may  include  CAI  (previously  discussed  in  Paragraph  5.5),  text,  films, 
sound-on-slide  packages,  or  other  materials.  If  CAI  materials  have  been  developed  for  a 
behavioral  objective,  study  of  these  materials  must  be  completed  by  those  airmen  who 
must  satisfy  knowledge  training  requirements  for  that  behavioral  objective.  When 
presented  with  an  opportunity,  an  airman  may  elect  to  complete  a  knowledge  training 
event.  To  accomplish  this  the  airman  may  log  onto  the  prototype  AOTS,  and  then  select 
from  the  menu  options  displayed  on  the  screen  those  that  permit  training  to  proceed.  The 
prototype  AOTS  will  then  perform  a  search  of  the  airman’s  ITR  and  select  the  knowledge 
training  event  that  should  be  trained  next.  If  the  event  is  a  knowledge  training  event, 
and  CAI  materials  are  available  for  that  event,  then  the  airman  will  be  presented  the 
materials  on  line.  The  ITR  search  and  event  selection  process  includes: 

a)  starting  from  the  top  of  the  ITR,  searching  for  the  first  task  having  a 
knowledge  training  event  that  has  been  assigned  for  training,  or  that  has  been 
scheduled  for  training  on  the  current  or  a  past  date,  and  selecting  that  event 
(assigned  for  training  means  the  supervisor  wants  the  task  trained  as  soon  as 
an  opportunity  is  presented,  but  an  event  could  not  be  scheduled  due  to  the 
lack  of  resources,  etc.);  and 

b)  if  no  event  has  been  assigned,  or  scheduled  on  the  current  or  a  past  date,  then, 
starting  from  the  top  of  the  ITR,  searching  for  the  first  task  that  has  a 
knowledge  training  event  reflected  as  not  completed,  and  selecting  that  event. 


If  no  event  has  been  selected  from  this  process  and  a  task  is  found  having  a  event 
scheduled  for  a  future  date,  the  process  will  be  terminated.  The  purpose  for  terminating 
the  process  is  to  ensure  that  no  training  are  evaluation  is  given  out  of  the  sequence 
established  by  the  immediate  supervisor.  If  a  supervisor  elects  to  have  an  event  that  has 
been  scheduled  for  a  future  date  start  sooner,  then  he/she  will  be  required  to  change  the 
schedule  to  the  current  date.  If  no  events  can  be  selected,  the  airman  will  be  so  informed. 
No  priority  for  behavioral  objectives  within  tasks  will  be  considered.  The  first 
incomplete  event  for  any  objective  within  the  established  sequence  becomes  the  candidate 
to  be  trained  next. 

As  indicated  above,  knowledge  training  materials  can  take  many  forms.  If  CAI 
materials  have  not  been  developed  for  an  objective  but  other  materials  have  been 
developed,  then  the  airmen  may  elect  to  study  whichever  materials  are  available. 
Materials  other  than  CAI  materials  will  be  completed  off  line.  When  an  airman  completes 
CAI  materials  the  ITR  will  automatically  be  updated  to  show  the  corresponding 
knowledge  training  event  as  completed.  When  materials  are  taken  off  line  airmen  must 
inform  the  AOTS  when  study  of  the  materials  has  been  completed.  This  will  be 
accomplished  by  responding  to  prompts. 

Again,  when  training  materials  have  not  been  developed  for  a  behavioral  objective 
ensuring  that  an  airman  acquires  the  required  knowledge  and  skills  is  ultimately  the 
responsibility  of  the  immediate  supervisor. 
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6.5  Evaluating  Training 

The  prototype  AOTS  implements  the  Air  Force  Inspector  General’s  recommendation 
to  provide  for  more  frequent,  valid,  and  reliable  evaluation  of  AFS  task  proficiency. 
This  is  accomplished  by  providing  the  capabilities  to  measure  airmen  acquisition  of 
requisite  knowledge,  literacy  requirements,  and  skills  appl:~*ble  to  AFS  tasks;  and  to 
evaluate  airmen  performance  of  AFS  tasks.  Certain  Ancillary  Training  and  Additional 
Duty  Training  courses  and  Contingency  Tasks  also  require  testing  for  completion. 
However,  these  will  not  be  tested  within  the  prototype  AOTS,  since  the  agencies 
responsible  for  administering  these  requirements  are  not  participating  in  the  SLT&E  of 
the  prototype  AOTS. 

The  development  of  evaluation  materials  to  be  used  within  the  prototype  AOTS  to 
measure  requisite  knowledge,  skills,  and  performance  was  previously  discussed  in 
Paragraph  5.3.  These  evaluation  materials  may  be  used  for  both  pre-training  testing  and 
for  post-training  testing.  Testing  prior  to  conducting  training  is  accomplished  when  it  is 
necessary  to: 

a)  determine  if  an  airman  already  possesses  the  knowledge  and  skills  required  to 

perform  a  specific  task,  or  is  capable  of  performing  that  task;  or 

b)  determine  where  training  on  a  specific  task  should  start  should  an  airman  not 

possess  the  knowledge  and  skills  required  to  perform  a  task,  or  be  incapable  of 
performing  that  task. 

Testing  an  airman  after  training  has  been  conducted  is  necessary  to  determine  if  the 
airman  has  acquired  the  knowledge  and  skills  required  to  perform  a  specific  task,  or  is 
capable  of  performing  the  task.  The  results  of  post-training  tests  arc  also  analyzed  to 
determine  the  validity,  comprehensiveness,  and  effectiveness  of  both  the  training  and  the 
tests. 


To  understand  how  evaluation  materials  are  to  be  used  within  the  prototype  AOTS 
requires  an  understanding  of  the  task  hierarchy  and  of  behavioral  objectives.  These  are 
discussed  in  Paragraphs  5.1  and  5.2,  respectively.  Evaluation  materials  have  been 
developed  at  the  behavioral  objective  level.  As  stated  in  Paragraph  5.2  there  are  two 
type'  of  behavioral  objectives  used  in  the  prototype  AOTS: 

a)  Terminal  Behavioral  Objectives,  which  correspond  to  tasks;  and 

b)  Supporting  Behavioral  Objectives,  which  correspond  to  subtasks. 

Generally,  two  types  of  tests  are  developed  for  terminal  and  supporting  behavioral 
objectives.  These  are: 

a)  a  knowledge  test  that  is  made  up  of  instructions  for  administering  the  test, 
and  of  symbolic  test  items  that  measure  requisite  knowledge  and  task  literacy 
(if  required);  and 

b)  a  performance  test  that  is  made  up  of  instructions  for  administering  the  test, 
and  of  checklists  that  are  used  to  rate  observed  performance  or  a  completed 
product. 
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The  exception  to  this  is  where  symbolic  tests  must  be  used  exclusively  because  task 
performance  cannot  be  measured  due  to  the  infrequency  of  occurrence,  etc.  Decisions 
relevant  to  whether  a  task  would  be  evaluated  only  at  the  terminal  behavioral  objective 
level,  only  at  the  supporting  behavioral  objective  level,  or  at  both  the  terminal  and 
supporting  behavioral  objective  levels  were  made  as  a  part  of  determining  evaluation 

strategies,  which  was  discussed  in  Paragraph  5.3.  All  tests  developed  for  a  behavioral 

objective  must  be  completed  before  that  objective  is  considered  to  be  completed.  All 
behavioral  objectives  for  a  task  must  be  completed  before  an  airman  is  considered  as 
eligible  for  qualification  certification. 

When  an  airman’s  ITR  is  updated  to  show  that  the  knowledge  training  for  a 
behavioral  objective  has  been  completed,  the  prototype  AOTS  will  perform  a  search  to 
determine  if  a  knowledge  test  has  been  developed  for  that  objective.  If  a  knowledge  test 
is  available  then  the  ITR  will  automatically  be  changed  to  show  a  requirement  to 
complete  the  test.  The  same  process  will  occur  when  an  airman’s  ITR  is  updated  to  show 
that  performance  training  for  an  objective  has  been  completed.  Knowledge  tests  will 
normally  be  taken  on  line.  They  may,  however,  be  printed  for  off-line  administration  by 
an  immediate  supervisor,  a  training  manager,  or  a  designated  evaluator.  Performance 
tests  used  within  the  prototype  AOTS  will  always  bs  administered  off  line.  An  airman 

may  be  administered  a  knowledge  test  on  line  by  going  through  the  process  outlined  in 

Paragraph  6.4,  if  the  event  selected  is  a  knowledge  ev  iluation  event. 


6.6  Recording  Trainee  Progress 

The  status  of  airmen  progression  in  AFS  task  events  and  other  training  requirement 
events  is  maintained  on  ITR  lists.  Status  includes:  required,  assigned,  scheduled,  or 
completed.  The  history  of  AFS  task  certifications  and  the  completions  of  other  training 
are  maintained  in  ATRs.  These  records  were  previously  discussed  in  Paragraphs  6.1  and 

6.2. 


When  training  requirements  are  entered  onto  ITRs,  and  no  other  status  is  reflected, 
they  are  simply  considered  as  required  to  be  completed  at  some  future  time.  When  a 
requirement  must  be  completed  at  the  earliest  possible  opportunity,  but  an  evcnt(s)  for  the 
requirement  cannot  be  scheduled,  the  responsible  person  should  change  the  status  of  the 
event(s)  to  reflect  assigned.  When  an  event  for  a  requirement  is  scheduled  the  status  will 
be  changed  automatically  to  reflect  scheduled.  When  an  AFS  task  knowledge  training 
event  is  completed  on  line  the  status  will  automatically  be  updated  to  reflect  completed. 
When  an  airman  studies  knowledge  training  materials  off  line,  and  responds  to  system 
prompts  that  study  was  completed,  the  status  of  the  corresponding  event  will 
automatically  be  changed  to  reflect  completed.  When  knowledge  training  materials  arc 
not  available  to  support  a  knowledge  training  event,  the  immediate  supervisor  is 
responsible  to  ensure  the  training  is  provided,  and,  therefore,  should  update  the  event 
when  training  is  completed.  Since  all  AFS  task  performance  training  events,  and  other 
training  events,  arc  conducted  off  line,  these  too  should  be  updated  by  the  responsible 
persons  when  training  is  completed.  The  status  of  knowledge  evaluation  events,  when 
tests  are  administered  on  line  and  passed,  is  automatically  changed  io  completed.  The 
status  of  knowledge  and  performance  evaluation  events,  when  tests  are  administered  off 
line,  passed,  and  read  into  the  system  via  optical  mark  readers,  is  automatically  changed 
to  completed.  If  the  status  of  training  and  evaluation  events  that  is  not  updated 
automatically  is  not  maintained  by  responsible  persons,  then  training  progress  cannot  be 
correctly  reported. 


6.7  Performing  Quality  Control  Evaluations 


Another  recommendation  resulting  from  the  aforementioned  Functional  Management 
Inspection  (FMI),  is  to  place  more  emphasis  on  the  quality  of  training  programs.  Within 
the  prototype  AOTS,  the  quality  of  training  programs  within  workcenters,  units,  and  the 
training  system  as  a  whole,  will  be  assessed  on  a  recurring  basis.  This  will  be 
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accomplished  through  evaluating  airman  performance  of  AFS  tr.sks  after  qualification  has 
been  certified,  and  providing  data  reflecting  the  evaluation  results  to  appropriate 
managers.  The  evaluations  are  referred  to  as  QC  evaluations.  Capabilities  are  provided 
to  generate  QC  evaluation  events  automatically  or  manually. 

The  prototype  AOTS  features  a  program  that  will  automatically  generate  QC 
evaluation  events,  at  specified  time  intervals.  The  system  will  select  the  tasks  to  be 
evaluated,  the  airmen  to  be  evaluated,  and  the  evaluators.  A  task  is  selected  only  if 
performed  at  Bergstrom  AFB  or  Ellington  ANGB,  and  if  the  values  for  the  task  factors 
(percent  of  members  performing  and  training  emphasis)  equal  or  exceed  specified 
percents.  An  airman  to  be  evaluated  is  selected  only  if  he/she  has  been  certified  on  the 
selected  task  within  the  past  specified  number  of  days,  and  if  he/she  has  not  been  QC 
evaluated  on  the  task  previously.  An  evaluator  is  selected  only  if  he/she  has  been 
certified  as  qualified  on  the  task  selected,  is  not  the  trainer  of  record,  did  not  certify  that 
the  airman  was  qualified  on  the  task  to  be  evaluated,  and,  obviously,  if  he/she  is  not  to 
be  evaluated  on  the  task.  Once  all  selections  have  been  made,  the  system  will  then 
automatically  schedule  the  event,  and  generate  a  schedule  for  off-line  distribution. 

In  instances  where  a  commander,  training  manager,  or  supervisor  wants  QC 
evaluation  event  generated  for  a  specific  task  or  airman,  the  QC  administrator  may 
interface  with  the  prototype  AOTS  to  make  appropriate  selections,  and  then  to  schedule 
the  event.  The  QC  administrator  will  be  prompted  to  specify  the  identification  number 
of  the  task  to  be  scheduled,  the  social  security  number  of  the  airman  to  be  evaluated,  the 
social  security  number  of  the  evaluator,  the  time  and’  place  for  the  event,  and  enter 
instructions,  if  required. 


6.8  Generating  Training  Reports  and  Notices 


The  quality  f  r.aining  under  the  purview  of  the  prototype  AOTS  is  further 
emphasized  througn  ,ne  generation  and  distribution,  to  appropriate  managers,  of  both 
standardized  a^d  ad  hoc  training  effectiveness  reports.  Standardized  reports  reflect 
training  progress  data  for  workcenters,  units,  and  the  prototype  AOTS  as  a  whole.  Ad  hoc 
reports  may  be  requested  from  the  system  administrator.  The  types  of  data  to  be  included 
in  ad  hoc  reports,  and  report  formats,  will  be  determined  by  the  requestor.  The  system 
also  gen  ;rates  notices  of  training  actions  required  when  changes  in  personnel  status  occur. 
These  notices  will  be  distributed  to  agencies  responsible  for  actions.  The  types  of 
standardized  reports  and  notices  produced  from  the  system  are  reflected  in  Figures  6-3 
and  6-4. 


•  Unit  Position  Qualification  Status  Summary 
■Unit  Upgrade  Training  Status  Report 

■  Workcenter  Position  Qualification  Status  Summary 
•Workcenter  Upgrade  Training  Status  Summary 

■  Individual  Position  Qualification  Status  Summary 
■Individual  Upgrade  Training  Status  Summary 
■Impacts  Caused  by  Personnel  Losses 

■  Base  Position  Qualification  Status  Summary 

■  Base  Upgrade  Training  Status  Summary 
•Test  Item  Analysis  Results 

■Base  Training  Resource  Availability /Utilization 
■Training  Event  Schedule 
•Training  Event  Completion 
•Training  Event  Cancelled 

■  Workcenter  Training  Event  Status  Summary 

•  Evaluation  Event  Schedule 
•Training  Development  Cost  Factors 
■Trainer  Performance  Data 


■  Task  Certification  Expiring 

■  CDC  Status  Report 

■  Failed  Quality  Cootrol  Evaluation/Recoratnendatlon 
for  Decertification 

■  Test  Requested  for  Off-Line  Testing 

■  Unit  Upgrade  Training  Roster 

■  Base  Upgrade  Training  Roster 

■  Workcenter  Upgrade  Training  Roster 

■  Individual  Trainlog  Notice 

■  CDC  Enrollment 

■  CDC  Completion 

•  Local/New  Task  Follow  Up  Requirement 

■  Unit  Training  Event  Status  Summary 

■  Base  Evaluator  Performance  Summary 

■  Unit  Event  Schedule  Failure 

■  Base  Event  Schedule  Failure  Summary 

■  Workcenter  Task  Coverage  Summary 


Figure  6*3  Standardized  Training  Reports 
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■  Change  in  DAFSC — New  OPTR  Required 

■  Trainee  TDY 

■  PRP  Disqualification 

■  Projected  Date  of  Separation  (DOS) 

■  Trainee  Hospitalized 

a  Trainee  Released  from  Hospital 

■  Trainee  Performing  Duty  out  of  CAFSC  (UGT) 
a  Trainee  Returned  to  Duty  in  CAFSC  (UGT) 


Figure  6-4  Personnel  Change  Notices 


Trainee  on  Leave 

Trainee  Returned  from  Leave 

Trainee  is  in  an  AWOL/Deserter  Status 

Trainee  Returned  to  Duty  from  an  AWOL/Deserter  Status 

Airman  in  Confinement 

Trainee  Performing  Duty  out  of  CAFSC  (PQT) 

Trainee  Returned  to  Duty  in  CAFSC  (PQT) 
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7  OPERATIONAL  SUPPORT 


The  AFHRL/IDD/OL-AK  AOTS  program  office,  located  at  Bergstrom  AFB,1  will  be 
providing  full  technical  and  management  support  to  all  AOTS  participants  during  SLT&E. 
HRL  will  be  providing  direct  and  indirect  support  to  all  users.  HRL  is  backed  up  by  the 
prime  development  contractor,  Douglas  Aircraft  Company,  and  the  site  management  and 
integrator  contractor  team.  Ball  Corporation,  Systems  Engineering  Division  and  BDM 
Corporation. 


7.1  Problem  Reporting 


Problems  encountered  while  using  the  prototype  AOTS  will  be  addressed  and  fixed 
by  AFHRL.  It  is  essential  that  all  problems  be  brought  to  the  attention  of  AFHRL. 
Routine  types  of  problems  can  be  handled  by  either  a  written  request  or  by  telephone. 
All  emergency  problems  that  will  have  an  effect  on  a  workcenter’s  training  should  be 
handled  by  telephone.  AFHRL  has  set  up  a  "hot  line"  telephone  at  Bergstrom  AFB  so  that 
a  user  can  call  in  problems  at  any  time  (day  or  night,  weekends,  etc.).  A  phone  answering 
machine  will  be  in  operation  when  the  1ST  and  contractor  personnel  are  away.  Your 
problem  will  be  answered  as  soon  as  possible  when  personnel  return. 


7.2  Hot  Line  Support 


The  hot  line  number  is  369-HELP  (AUTOVON  number  8-685-HELP)  or,  if  you  are 
located  on  Bergstrom  AFB,  simply  dial  HELP  (or  4357).  Any  problem,  how  to  use, 
hardware,  software,  training  materials,  procedures,  etc.,  are  all  referred  to  the  hot  line  for 
fast  service. 


The  regular  AFHRL/IDD/OL-AK  program  management  office  number  is  AU- 
TOVON  8-685-2667.  The  1ST  number  (at  building  1808)  is  8-685-3070. 

7.3  AFHRL/IDD/OL-AK  AOTS  Program  Organization 


Figure  7-1  provides  an  overview  of  the  AFHRL  AOTS  program  organization  at 
Bergstrom  AFB.  The  program  chief  is  Major  J.  Blackhurst  (IDD/OL-AK).  The  prime 
development  contractor  is  Douglas  Aircraft  Corporation,  a  division  of  McDonnell  Douglas 
Corporation  and  the  site  management  and  integrator  contractor  is  a  team  headed  by  Ball 
Corp.,  Systems  Engineering  Division  and  BDM  Corporation. 


Tin'  API.?!.  Opera l  in-*.  I. neat  ion  at  Bergstrom  AFB,  Texas, 


has  been  digest 
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CONTRACTOR 


Figure  7-1  The  AOTS  Program  Organization  with  AFHRL 
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8  GLOSSARY  OF  TERMS 


ACRONYM 

DEFINITION 

A/C 

Aircraft 

AF 

Air  Force 

AF/DPP 

Air  Force/Director  Personnel  Programs 

AFB 

Air  Force  Base 

AFHRL 

Air  Force  Human  Resources  Laboratory 

AFHRL/IDD/ 

OL-AK. 

Air  Force  Human  Resources  Laboratory,  IDD/OL-AK, 
AOTS  branch  within  the  laboratory  operating  at 
Bergstrom  AFB 

AFMPC 

Air  Force  Military  Personnel  Center 

AFRES 

Air  Force  Reserves 

AFS 

Air  Force  Speciality 

AFSC 

Air  Force  Systems  Command 

AFSC 

Air  Force  Speciality  Code 

ANG 

Air  National  Guard 

ANGB 

Air  National  Guard  Base 

AOTS 

Advanced  On-the-job  Training  System 

asvab 

Armed  Services  Vocational  Aptitude  Battery 

ATR 

Airman  Training  Record 

BDM 

BDM  Corporation 

bsed 

Ball  Corporation,  Systems  Engineering  Division 

bo 

Behavioral  Objective 

CAFSC 

Career  Air  Force  Speciality  Code 

CAI 

Computer  Assisted  Instruction 

CAMS 

Core  Automated  Maintenance  System 

CAT 

Computer  Assisted  Training 

CBPO 

Consolidated  Base  Personnel  Office 

CDC 

Career  Development  Courses 

CDTS 

Computer  Directed  Training  System 

DCM 

Deputy  Commander  for  Maintenance 

DOB 

Date  Of  Birth 

DOS 

Date  Of  Separation 

ECI 

Extension  Course  Institute 

Ext. 

Extension 

FAC 

Functional  Account  Code 

FIG 

Fighter  Interceptor  Group 
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FMI 

GPTR 

HSD 

HRL 

ID 

ISD 

ISS 

1ST 

ITR 

MAJCOM 

MGMT 

MTL 

OJT 

OMR 

OPTR 

OTGs 

OTR 

PAS 

PCIII 

PECs 

PDS 

PECs 

PQT 

PRP 

PUBs 

QC 

R&D 

SHM 

SLT&E 

SMEs 

SOAs 

SPAS 

TAC 

TDY 

TFG 

TIB 

TRW 

TX 

UMDs 

USAFOMC 


Functional  Management  Inspection 
Generic  Position  Task  Requirements 

Human  Systems  Division  (of  Air  Force  Systems  Com¬ 
mand) 

Human  Resources  Laboratory 
Identification 

Instructional  Systems  Development 

Instructional  Support  System 

Instructional  Support  Team 

Individual  Training  Requirements 

Major  Command 

Management 

Master  Task  List 

On-the-Job  Training 

Optical  Mark  Reader 

Operational  Position  Task  Requiremcn  > 

Oral  Test  Guides 

Other  Training  Requirements 

Personnel  Accounting  Symbol  System 

Personnel  Concept  III 

Performance  Evaluation  Checklists 

Personnel  Data  System 

Performance  Evaluation  Checklists 

Position  Qualification  Training 

Personnel  Reliability  Program 

Publications 

Quality  Control 

Research  and  Development 

Short  Haul  Modem 

System  Level  Test  and  Evaluation 

Subject  Matter  Experts 

Separate  Operating  Agencies/Activities 

Security  Police  Automated  System 

Tactical  Air  Command 

Temporary  Duty 

Tactical  Fighter  Group 

Test  Item  Bank 

Tactical  Reconnaissance  Wing 

Texas 

Unit  Manning  Documents 

United  States  Air  Force  Occupational  Measurement 
Center 
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9  APPLICABLE  DOCUMENTS 


Document: 


Bergstrom  AFB;  Site  Preparation  Requirements  and  Equipment  Installation  Plan, 
prepared  by  Douglas  Aircraft  Company,  division  of  McDonnell  Douglas 
Corporation  under  contract  F33615-84-C-0059  for  the  USAF;  dated:  28  January 
1988 


Ellington  ANGB;  Site  Preparation  Requirements  Plan,  prepared  by  Douglas  Aircraft 
Company,  division  of  McDonnell  Douglas  Corporation  under  contract 
F33615-84-C-0059  for  the  USAF;  dated:  January  26,  1988 

Facilities  Requirements  Plan  for  the  Transition  of  the  Advanced  On-the-job 
Training  System  to  Ellington  Air  National  Guard  Base,  prepared  by  Ball 
Corporation,  Systems  Engineering  Division/BDM  Corporation  under  contract 
F336 15-84-0070  by  BDM  Corporation  for  the  USAF;  dated:  16  October  1987 


Master  Test  Plan;  prepared  by  Douglas  Aircraft  Company,  division  of  McDonnell 
Douglas  Co-poration  under  contract  F33615-84-C-0059  for  the  USAF;  dated:  21 
August  1987,  Revision  2.0 


Interim  Technical  Report;  prepared  by  Douglas  Aircraft  Company,  division  of 
McDonnell  Douglas  Corporation  under  contract  F33615-84-C-0059  for  the 
USAF;  dated:  December  30,  1986 


Maintenance  Plan;  prepared  by  Douglas  Aircraft  Company,  division  of  McDonnell 
Douglas  Corporation;  dated:  October  28,  1986,  Version  3.0 

Computer  Programming  Standards  Document;  prepared  by  Douglas  Aircraft 
Company,  division  of  McDonnell  Douglas  Corporation  under  contract 
F33615-84-C-0059  for  the  USAF;  dated:  15  September  1986 


System  Engineering  Management  Plan;  prepared  by  Douglas  Aircraft  Company, 
division  of  McDonnell  Douglas  Corporation  under  contract  F33615-84-C-0059 
for  the  USAF;  dated:  September  30,  1986,  Version  2.1 


Human  Engineering  Plan;  prepared  by  Douglas  Aircraft  Company,  division  of 
McDonnell  Douglas  Corporation  under  contract  F33615-84-C-0059  for  the 
USAF;  dated:  20  January  1987 


Reliability  Program  Plan;  prepared  by  Douglas  Aircraft,  a  division  of  McDonnell 
Douglas  Corporation  under  contract  F33615-84-C-0059  for  the  USAF;  dated: 
September  3,  1986 


Maintainability  Program  Plan;  prepared  by  Douglas  Aircraft,  a  division  of 
McDonnell  Douglas  Corporation  under  contract  F33615-84-C-0059  for  the 
USAF 
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Configuration  Management  Plan;  prepared  by  Douglas  Aircraft,  a  division  of 
McDonnell  Douglas  Corporation  under  contract  F33615-84-C-0059  for  the 
USAF;  dated:  December  19,  1986,  Version  1.01 


70S647000;  AOTS  System  Specification  (Al);  prepared  by  Douglas  Aircraft  Company, 
division  of  McDonnell  Douglas  Corporation  under  contract  F33615-84-C-0059; 
dated:  May  5,  1985 

70S647100;  Management  Subsystem  Prime  Item  Specification  (Bl);  prepared  by 
Douglas  Aircraft  Company,  division  of  McDonnell  Douglas  Corporation  under 
contract  F33615-84-C-0059;  dated:  March  13,  1987 

70S647300;  Evaluation  Subsystem  Prime  Item  Specification  (Bl);  prepared  by 
Douglas  Aircraft  Company,  division  of  McDonnell  Douglas  Corporation  under 
contract  F33615-84-C-0059;  dated:  November  7,  1986 


70S647200;  Training  Development  and  Delivery  Subsystem  Prime  Item  :  pecification 
(Bl);  prepared  by  Douglas  Aircraft  Company,  division  of  McDor.  lell  Douglas 
Corporation  under  contract  F33615-84-C-0059 

70S647400;  Computer  Support  Subsystem  Prime  Item  Specification  (Bl);  prepared  by 
Douglas  Aircraft  Company,  division  of  McDonnell  Douglas  Corporation  under 
contract  F33615-84-C-0059 


70S647401;  Critical  Item  Development  Specification  for  the  Hardware  Component  of 
the  Advanced  On-the-job  Training  System;  prepared  by  Douglas  Aircraft 
Company,  division  of  McDonnell  Douglas  Corporation;  dated:  1988,  Version  2.0 

70S647201;  Training  Development  Component  (B2)  Specification;  prepared  by 
Douglas  Aircraft,  a  division  of  McDonnell  Douglas  Corporation  under  contract 
F336 1 5-84-C-0059  for  the  USAF 

70S647202;  Training  Delivery  Component  (B2)  Specification;  prepared  by  Douglas 
Aircraft,  a  division  of  McDonnell  Douglas  Corporation  under  contract 
F336 1 5-84-C-0059  for  the  USAF 


70S64741 1;  Management  Subsystem  CPCI  (B5)  Specification;  prepared  by  Douglas 
Aircraft,  a  division  of  McDonnell  Douglas  Corporation  under  contract 
F33615-84-C-0059  for  the  USAF 


70S647413;  Evaluation  Subsystem  CPCI  (B5)  Specification;  prepared  by  Douglas 
Aircraft,  a  division  of  McDonnell  Douglas  Corporation  under  contract 
F33615-84-C-0059  for  the  USAF 


70S6474I4;  System  Support  CPCI  (B5)  Specification;  prepared  by  Douglas  Aircraft,  a 
division  of  McDonnell  Douglas  Corporation  under  contract  F33615-84-C-0059 
for  the  USAF 


70S647412;  Training  Development  and  Delivery  Subsystem  B5  Specification; 
prepared  by  Douglas  Aircraft,  a  division  of  McDonnell  Douglas  Corporation 
under  contract  F33615-84-C-0059  for  the  USAF 


70S647411;  Management  CPCI  (C-5)  Specification;  prepared  by  Douglas  Aircraft,  a 
division  of  McDonnell  Douglas  Corporation  under  contract  F33615-84-C-0059 
for  the  USAF 
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70S647413;  Evaluation  CPCI  (C-5)  Specification;  prepared  by  Douglas  Aircraft,  a 
division  of  McDonnell  Douglas  Corporation  under  contract  F3361S-84-C-0059 
for  the  USAF 


70S647414;  System  Support  CPCI  (C-5)  Specification;  prepared  by  Douglas  Aircraft, 
a  division  of  McDonnell  Douglas  Corporation  under  contract  F33615-84-C-0059 
for  the  USAF 


70S647412;  Training  Development  and  Delivery  CPCI  (C-S)  Specification;  prepared 
by  Douglas  Aircraft,  a  division  of  McDonnell  Douglas  Corporation  under 
contract  F33615-84-C-0059  for  the  USAF 


Contract  F33615-84-C-0059;  United  States  Air  Force  and  Douglas  Aircraft  Company, 
a  division  of  McDonnell  Douglas  Corporation;  dated:  1  August  1985 


MIL-STD-490,  30  OCT  1968,  Military  Standard  Specification  Practices 

MIL-STD-483,  Notice  2,  21  Mar  1979,  Configuration  Management  for  System, 
Equipment,  Munitions,  and  Computer  Programs 

MIL-STD-1815A,  22  Jan  1983,  Ada  Program  Language 


AFHRL-TR-75-83,  "On-The-Job  Training  in  the  Air  Force"  by  R.W.  Stephenson  and 
J.  R.  Burkett,  dated:  December  1975 


PN  76-269,  Air  Force  Inspector  General  Functional  Management  Inspection  (FMI) 
report  titled:  "On-The-Job  Training  in  the  Air  Force",  dated:  April  1977 


AFHRL-TP-83-54,  'Integrated  Training  System  for  Air  Force  On-The-Job  Training: 
Specification  Development",  dated:  March  1984 


STRUCTURED  SYSTEMS  ANALYSIS:  TOOLS  and  TECHNIQUES,  Gane,  C.  and 
Sarson,  T.,  McDonnell  Douglas  Corporation,  1982 


HIPO-  A  Design  Aid  and  Documentation  Technique,  IBM,  1975 


AFR  50-23,  Regulation,  On-the-Job  Training 
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10  APPENDIX  A  -  ATR  Screen  Displays 


This  appendix  has  representative  screen  displays  from  the  ATR.  Various  users  will 
find  information  displayed,  during  SLT&E,  taken  from  the  AOTS  data  base.  These  screen 
representations  will  give  readers  of  this  document  a  feel  of  what  the  system  will  actually 
display. 


Training  Rtcord  Manager 
Personnel  Data 
Personal  Information 


1 .  Name: 

2.  D08: 

S.  Enlistment  Category: 

8.  Primary  .AfSC: 

10.  Secondary  AFSC: 

12.  Third  AfSC: 


SSAN: 

3.  TAfMSD:  4.  DOS: 

6.  Grade:  7.  Projected  Grade: 

AFSC  Information 

9.  Fourth  AFSC: 

11.  Control  ATSC: 
13.  Duty  AFSC: 

Position  Information 


14.  Number:  15.  Duty  Status: 

17.  Title: 

19.  Date  Arrived  Station: 

21.  Date  assigned  to  workcenter: 


16.  Record  Status: 
18.  Phone  Ext: 

20.  Office  Symbol: 
22.  Duty  Type: 


23.  Number  : 

26.  Projected  PAS: 
29.  Nomenclature: 
31.  Installation: 


Organization  Information 

24.  Type:  2S.  Location  ID: 

27.  PAS:  28.  FAC: 

30.  Detachment  •: 
32.  Printer  ID: 


Training  Record  Manager  -  Personal,  Position,  and  Organization  Information 


ATR  Manager 

General  Training  History  Ur 


1.  Training  Status  Code: 

2.  Date  Fotered/CompJeted/Withdrawn  UGT: 

3.  Date  Entered/Compieted/Withdrawn  Training: 

4.  Date  Entered/Completed  Position  Qualification  Training: 

5.  Date  Initially  Entered  Re-Training: 

6.  High  Education  Level: 

7.  Secondary  Education  Level: 

8.  ASVAB  Administrative  Score: 

9.  ASVAB  Mechanical  Score: 

10.  ASVAB  Electronics  Score: 

]  1 .  ASVAB  General  Score: 

12.  Reading  Achievement  Score  1: 

1 3.  Date  Test  l : 

14.  Reading  Achievement  Score  2: 

15.  Date  Tested  2: 

16.  Typing  Teat  Score: 

17.  Date  of  Typing  Teat: 


Training  History  Information 
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Training  Record  Manager  ■  Professional  Military  Education 


Training  Record  Manager  •  Other  Formal  Training  Information 
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Training  Record  Manager  •  ECI/CDC  Training 


Training  Record  Manager  *  Ancillary  History  Information 


ACTS  Operational  Guide 


Training  Record  Manager  -  Additional  Duty  History  information 


Training  Record  Manager  -  Contingency  History  Information 
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Training  Record  Manager  -  AOTS  Task  Training  Information 
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1 1  APPENDIX  B  -  The  Prototype  AOTS  Configuration 


The  purpose  of  this  appendix  is  to  provide  details  on  various  aspects  of  the 
prototype  Advanced  On-the-job  Training  System’s  (AOTS)  Computer  Support  Subsystem. 
The  system’s  architecture  will  be  discussed,  the  Bergstrom  AFB  and  Ellington  ANGB 
installations,  how  the  hardware  was  selected  along  with  selected  specifications/require¬ 
ments,  and  the  software  architecture.  It  is  not  the  intent  of  this  appendix  to  discuss 
AOTS  from  a  training  system  aspect. 


11.1  System  Architecture 

The  prototype  Advanced  On-the-job  Training  System  (AOTS)  is  based  on  a 
centralized  computer  architecture.  The  AOTS  data  base  is  the  central  location  where  all 
information  is  stored.  Figure  11-1  provides  a  high  level  overview  of  the  prototype 
architecture.  The  central  computer,  a  Digital  Equipment  Corporation  VAX  8650 
computer,  is  located  at  AFHRL/TS  at  Brooks  AFB.  High  speed  digital  communications 
link  Bergstrom  AFB  and  Ellington  ANGB  to  the  VAX  computer. 

Zenith  Z-248  Personal  Computers  (PCs)  are  used  for  the  AOTS  terminals.  Only 
limited  local  processing  occurs  at  the  PC.  There  will  be  located  at  various  PC  terminals  a 
dot  matrix  printer  (Alps  P2000G)  and  an  optical  mark  sense  reader.  The  latter  is  a  device 
for  reading  in  test  forms.  AOTS  offers  the  capability  of  administering  tests  off-line  and 
the  results  are  entered  into  the  computer  with  a  special  mark  sense  form.  These  forms  are 
in  common  use  currently  in  the  Air  Force. 
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Figure  11-1  The  Prototype  AOTS  Hardware  Configuration 


Table  11-1  provides  an  overview  of  each  major  type  of  equipment  used  and  the  total 
quantities  of  each  in  the  prototype  type  AOTS.  The  VAX  computer  system  was  an 
existing  Air  Force  asset  used  with  the  remainder  of  the  equipment  either  purchased  or 
leased  specifically  for  AOTS. 


i 
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Model/Part 

Number 

QTY 

Description 

Manufacture  | 

VAX  8650 

i 

Computer  System  (Located  AFHRL/TS  at  Brooks 
APB,  TX) 

Digital  Equipment  1 
Corporation  | 

56  Kilobaud  Communications 
Lines 

3 

Two  from  Brooks  AFB,  TX  to  Bergstrom  APB.  TX 
and  one  from  Brooks  AFB  to  EKngton  ANGB,  TX. 

Clay  Desta  | 

56KB  Multiplexers  and 

Modems 

4  Sets 

On*  set  located  located  at  Brooks  AFB.  two  sets  at 
Bergstrom  AFB.  end  one  set  at  Elngton  ANGB. 

Infotron  1 

Zenith  ZWX-246-52 

85 

Personnel  Computer 

Zenith  1 

Zenith  ZVM-1380 

85 

Color  Monitor  (or  Z-248 

Zenith  I 

Zenith/Alps  P2000G 

22 

Dot  Matrix  Printer 

Zenith/Alps  jj 

Model  5200 

17 

Optical  Merit  Reader 

Scantron  H 

LaserJet  500  Plus 

5 

Laser  Printer  and  font  cartridges 

Hewlett  Packard 

LaserJet  II 

2 

Laser  Printer  and  font  cartridges 

Hewlett  Packard 

Ehemet  Local  Area  Network 

1 

Ethernet  Local  Area  Network  (LAN) 

3Com 

P  m  Compiler 

2 

Software,  Ada  compiler,  with  extended  memory 
board,  for  PC 

Aisys 

C  ikx  Printer.  Okidata  20 

3 

Color  printer  with  paraftei  and  serial  interfaces 

Okimate 

t  jitizer  Tablet 

2 

Digitizer  20  X  20  inch 

Summagraphics 

Bit  Pad 

2 

Digitizer  Pad 

Table  11-1  AOTS  Prototype  Equipment  Types  and  Quantities 


1 1 .2  Communications 


The  high  speed  communication  subsystem  used  (Figure  1 1-1)  is  three  leased  56 
kilobaud  lines  from  Clay  Desta.  The  link  between  the  central  cities  of  San  Antonio, 
Austin,  and  Houston  is  with  fiber  optics.  The  final  link  from  Clay  Desta’s  central  offices 
to  the  bases  uses  conventional  copper  telephone  lines  with  special  signal  repeaters. 
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11.3  Bergstrom  AFB  Installation 


The  AOTS  communication  configuration  for  Brooks  AFB  ami  Bergstrom  AFB  is 
provided  in  Figure  11-2.  The  topology  used  resembles  a  ring  network.  It  is  not  a  true 
ring  type  local  area  network  but  does  have  some  key  features  that  resemble  a  ring. 
Redundancy  is  provided  to  the  users:  should  one  of  the  two  major  lines  fail,  the  load  will 
be  switched  to  the  other.  The  system  also  has  auto  load  level  so  that  response  level  will 
be  maintained  at  the  highest  possible  level. 


■  RING  TYPE  TOPOLOGY  USED 

■  REDUNDANCY  AND  CAPACITY  (AUTO  LOAD  LEVELING) 


Figure  11-2  The  Bergstrom/Brooks  AFB  Communication  System 


Table  11-2  provides  a  breakdown  on  all  equipment  that  will  be  used  on  Bergstrom 
AFB.  Figure  11-2  provides  a  detail  communication  diagram  for  Bergstrom  AFB.  Building 
1101  is  the  Bergstrom  Communication  Squadron  main  building  and  the  largest  multiplexer 
will  be  located  there.  It  is  from  this  building,  using  existing  Bergstrom  AFB 
communication  lines,  that  the  various  users  will  be  connected  to  the  central  computer. 

The  high  speed  lines  going  to  Buildings  1808  (specially  built  by  AFHRL  for  this 
project)  and  Building  428  also  have  multiplexers.  During  the  development  phase  of  AOTS 
(Phase  II),  a  single  leg  of  the  communication  system  diagram  in  Figure  11-3  was  used 
connecting  these  buildings  to  the  VAX  computer  at  Brooks  AFB.  The  multiplexers  will 
remain  in  use  throughout  the  contract.  Building  1808  is  used  for  the  AFHRL/IST 
personnel  who  are  developing  the  data  base.  Building  428  is  the  location  for  the  Douglas 
Aircraft  Company  personnel  who  are  developing  the  prototype  AOTS. 
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Location  at 

Bergstrom  AFB 

Z-248 

OMR 

Printer 

Communication 

Lines 

JET  ENGINE  MAINTENANCE 

BLDG  1612 

2 

1 

1 

5 

BLDG  4529 

1 

0 

0 

2 

BLDG  4589  (RES.) 

2 

1 

1 

6 

A/C  MAINTENANCE 
BLOG  1609 
BLOG  4529 
BLOG  4515  (RES.) 


PERSONNEL 
BLDG  2202 
BLDG  206 
BLDG  4555  (RES.) 


BASE  OJT  MONITOR 
BLOG  2202 
BLDG  2210 


SECURITY  POLICE 
BLDG  W 
BLDG  .08 
BLDG  1204  (RES.) 


CHIEFS  TF  MAINTENANCE 
TRAINI-  3  STAFF 
BLDG  .501 
BLDG  4592  (RES.) 


AFHRL/OL-AK 
BLDG  T-1 
BLDG  1808 
BLDG  428 


TOTALS: 


Table  11-2  The  Bergstrom  Equipment  List  by  Organization  and  Building 


Figure  11-3  Bergstrom  Communication  Layout 
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11.4  Ellington  ANGB  Installation 


The  Air  National  Guard  (ANG)  was  added  to  the  prototype  AOTS  after  the  start  of 
the  current  AOTS  efforts.  Figure  11-4  illustrates  the  ANG  equipment  configuration.  The 
ANG  will  also  use  the  VAX  computer  at  Brooks  AFB  (AFHRL/TS)  same  as  the  active  and 
reserves  at  Bergstrom  AFB.  The  major  difference  is  in  the  amount  of  equipment.  The 
configuration  and  quantity  of  equipment  is  based  on  the  routine  ANG  UTA  events.  All 
of  the  equipment  used  at  Ellington  ANGB  is  of  the  same  type  as  used  at  Bergstrom  AFB. 
No  new  hardware  or  software  has  been  developed  for  the  ANG. 


Table  11-3  shows  the  equipment,  by  building  location,  that  will  be  used  by  the  Air 
National  Guard  at  Ellington  ANGB.  Table  11-4  is  the  total  equipment  procured  for  the 
ANG. 


Page  47 


AOTS  Operational  Guide 


Location  at 

Ellington  ANGB 

Z-248 

OMR 

Printer 

Communication  | 
Lines  g 

Buiding  1382 

4 

2 

4 

12 

Building  1293 

1 

1 

1 

3 

8uilding  1057 

2 

1 

2 

5 

BuSdkig  1193 

2 

1 

2 

5 

TOTAL: 

9 

5 

9 

25 

Table  11-3  The  Ellington  ANGB  Equipment  Distribution  by  Building 


PART 

NUMBER 

QTY 

(ea.) 

DESCRIPTION 

MANUFACTURE 

LOCATION 

REMARKS 

992/SU 

SM 632/2-790 

CL790/LM790V 

CA790/4M4 

LD8/5 

LD8/1 

CL790/LM600 

CA600/4-790 

1 

Control  Unit 

Expansion  Chassis 

Link  Modules 

Quad  Board  4  RS232  ports 

Short  Haul  Modem 

Short  Haul  Modem 

Link  board  for  SM632 

Quad  Board  for  SM632 

Infotron 

Infotron 

Infotron 

Infotron 

Infotron 

Infotron 

Infotron 

Infotron 

Ellington  ANGB 
Brooks  AFB 

Brooks  AFB 

Ellington  ANGB 

Ellington  ANGB 
Ellington  ANGB 

Brooks  AFB 

Brooks  AFB 

Allows  existing 
992/SU  at  Brooks 
to  be  used  for 
ANG  communica¬ 
tion  line. 

Allows  Bergstrom 
to  control  ANG 
communications. 

1  at  Ellington  is  a 
spare 

Links  SM632  and 
992/SU 

4  RS232  Ports 

DOS-56 

3 

56KB  Modem 

UDS 

2  at  Ellington 

1  at  Brooks 

1  at  Ellington 

ANG  is  a  spare 

SWTC-PVC4C24 

1000  Ft 

4  Wire  Cable 

Graybar 

3 !! !?  !??E  !  "  :■ 

PWC-02P24 

2000  Ft 

Cable  Twisted  "'air 

Graybar 

56  KB  digital  data 
line 

13 

(Months) 

56  Kilobaud  Leased  Line 

Clay  Desta 

Ellington  ANG  to 
Brooks  AFB 

A  leased  line  for 

12  months 

Houston  Loop 

13 

(Months) 

Local  Houston  Loop 

Clay  Desta 

Ellington  ANGB 

San  Antonio  Loop 

13 

Local  San  Antonio  Loop 

Clay  Desta 

Brooks  AFB 

5200 

5 

Optical  Mark  Sense  Reader 

Scantron 

Ellington  ANGB 

87F2866 

10 

Power  Strip 

Newark 

Ellington  ANGB 

5200 

5 

Cable 

Innovative 

Ellington  ANGB 

Scantron  to  PC 

6C/C 

23 

Cable.  RS232 

Innovative 

Brooks  AFB- 

VAX  to  MUX 

Table  11-4  -  Air  National  Guard  Equipment  List  for  SLT&E 
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I 


1 1 .5  Hardware  Selection  Process 


During  the  preliminary  design  phase  of  the  AOTS  (Phase  I),  trade  studies  were 
conducted  to  determine  what  equipment  would  be  needed  to  meet  the  AOTS  requirements. 
Hardware  and  some  software  trade  studies  were  conducted.  These  trade  studies  were 
broken  down  into  various  key  fields. 


11.5.1  Trade  Studies 

The  two  principal  hardware  trade  studies  focused  on  the  central  computer  and  the 
terminals.  Smaller  trade  studies  also  were  conducted  for  the  printers,  digitizer  tablets, 
and  optical  mark  sense  readers. 

The  trade  study  centered  on  three  vendors.  These  were  Digital  Equipment 
Corporation  (DEC),  Gould  Corporation,  Computer  Systems  Division,  and  Data  General 
Corporation  (DG).  Three  valid  computer  systems  were  determined  could  meet  the  AOTS 
requirements.  These  were,  by  rating  starting  with  the  highest  rated  to  lowest  acceptable, 
the  Gould  9000  system,  DEC  VAX  8650,  and  DG  MV2000.  AFHRL  could  provide  a  DEC 
VAX  8600  within  existing  inventory  which  could  be  upgraded  to  a  8650  at  minimum 
expenditure  of  funds.  Thus,  the  DEC  VAX  8650  was  selected.  The  key  characteristics  of 
the  central  computer  arc  provided  in  Table  11-5. 


COMPONENT 

REQUIREMENTS 

Computer 

Have  a  Validated  Ada  compiler 

32  Bit  Machine 

Virtual  Memory  addressing 

5  MIPS  (Phase  II)  Performance 

10  MIPS  (Phase  III)  Performance 

Separate  Floating  Point  Accelerator 

Cache  Memory  (minimum  of  8  kilobyfes) 

Vector  Priority  Interrupt  Structure 

Real  Time  Clock  and  Interval  Timer 

I/O  bandwidth  of  20  megabytes/second 

Virtual  Memory  Disk  Based  Operating  System 

Memory: 

64  Megabytes  of  main  memory 

Disk  Storage: 

1.3  Gigabytes  of  disk  storage  using  multiple  spindles 
and  controllers 

Magnetic  Tape: 

FIPS  PUB  25  compatible 

Line  Printer 

445  Lines  Per  Minute  with  a  96  character  set 

Communications: 

64  RS232C  ports  (minimum  of  48  required  Phase  II) 

124  RS232C  ports  (Phase  III) 

Physical  Considera¬ 
tions: 

Operate  using  conventional  power  115  -  220  VAC,  up  to  three  phases,  60  Hz 
Operate  on  conventional  air  conditioning  while  maintaining  72  degrees  F,  ♦/-  8 
degrees  F 

Meet  FCC  emissions  standards  (Part  15  Subpart  J,  Level  A) 

Reliability/ 

Maintainability: 

MTBF  of  600  hours  minimum  (goaf 

MTTR  of  1.5  hours  maximum  (goal 

Table  11-5  AOTS  Central  Computer  System  Requirements 


The  terminal  selection  first  had  to  resolve  some  fundamental  questions.  These  were: 
a)  Was  color  a  requirement? 
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b)  Was  graphics  a  requirement? 

c)  How  smart  or  intelligent  should  the  terminal  be? 

As  the  result  of  analysis  of  the  AOTS  prototype  requirements,  the  terminals  were 
determined  to  require  color,  medium  resolution  graphics,  and  should  be  capable  of  future 
growth.  Table  11-6  provides  an  overview  of  the  higher  priority  specifications.  The  key 
parameters  are: 

a)  16  Colors  for  both  text  and  graphics 

b)  600  X  350  graphic  resolution 

c)  Growth  to  an  intelligent  terminal 

d)  Ergono  nically  designed  for  ease  of  use  by  any  type  of  user 

e)  Highly  reliable 

Extensive  trade  studies  were  conducted  into  various  vendors  of  personal  computers, 
color  monitors,  graphic  capabilities,  ergonomics,  reliability,  and  cost.  It  was  also 
determined  that  the  prototype  AOTS  should  stay  on  the  leading  edge  of  technology 
without  taking  undo  risks.  This  led  to  the  decision  to  only  study  PCs  that  used  the  80286 
microprocessor.  This  eliminated  all  8088/8086/80186  designs  as  approaching  obsolescence. 
The  80386  machines  were  only  just  being  prototyped  and  were  considered,  for  the 
prototype,  as  too  risky.  It  was  further  decided  that  the  68000  microprocessor  based  PCs 
were  either  too  costly  or  too  limited  in  capability  (e.g.  no  color  capability).  Once  the 
80286  was  the  type  of  microprocessor,  it  was  determined  that  the  prototype  AOTS  should 
also  stay  as  close  to  standards  as  possible.  There  strong  indications  that  the  Air  Force  was 
leaning  toward  the  IBM  PC  AT  type  of  PC  for  meeting  its  existing  requirements.  Further, 
the  80287  numeric  coprocessor  should  also  be  added  to  all  prototype  AOTS  terminals. 

A  trade  study  was  conducted  on  IBM  AT  type  PCs.  The  study  was  reduced  to  IBM, 
Zenith,  Compaq,  Sperry,  Hewlett  Packard,  ITT,  and  AT&T.  The  study  determined  that  all 
met  or  exceeded  requirements  and  that  cost  or  other  factors  should  be  used  to  make  the 
final  determination.  At  the  time  the  trade  studies  were  being  concluded,  the  Air  Force 
issued  the  procurement  for  the  Zenith  Z-248  PCs.  Thus,  AOTS  selected  the  Z-248  for  the 
prototype  terminal. 


The  trade  studies  for  the  color  monitor  and  the  graphics  capability  also  closely 
mirrored  the  capabilities  offered  in  the  IBM  PC  AT.  These  included  the  existing 
monochrome  text  and  graphics  capability,  the  Color  Graphics  Adapter  (CGA)  standard. 
Extended  Graphics  Adapter  (EG\)  standard,  and  Professional  Graphics  Adapter  (PGA) 
standard.  The  CGA  was  inadequate  for  meeting  the  resolution  requirements  and  the  PGA 
was  considered  too  expensive.  The  EGA  standard  meets  AOTS  resolution  and  color 
requirements  and  was  also  part  of  the  standard  buy  of  Zenith  Z-248  personal  computers. 
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The  result  of  the  trade  studies  was  a  set  of  requirements.  These  requirements  arc 
provided  in  Tables  9-6  and  9-7  which  describe  the  AOTS  terminal  and  peripherals 
requirements. 


COMPONENT 


REQUIREMENTS 


Computer 

80286  microprocessor  at  8  MHz  i 

MS-DOS  3.1  or  higher  operating  system  1 

IBM  PC  AT  compatible  1 

QWERTY  detached  keyboard  AT  style  g 

One  Parallel  Printer  Port  (Centronics  compatible)  1 

Two  RS232C  Serial  Ports  capable  ot  9600  baud  rates  f 

IBM  Standard  Graphics  Board  (EGA)  D 

480  X  360  resolution  (or  the  graphics 

Non-Interlaced  output 

RS-170  compatible 

Minimum  of  16  color  palette 

96  character  set  with  true  descenders 

9x11  character  cell  or 

8  x  14  character  cell  minimum 

Capable  of  supporting  a  light  pen 

Capable  of  supporting  a  mouse 

Capable  of  supporting  interactive  video  controllers  and  graphics /text  overlay 
capability  in  color 

Meet  FCC  Part  15,  Subpart  J,  Class  A  emissions  (minimum) 

Memory: 

Minimum  of  512  kilobytes  of  memory 

Expandable  to  4  megabytes 

Disk  Storage: 

360  Kilobaud  IBM  standard  floppy  disk 

20  Megabyte  hard  (Winchester)  disk 

Access  time  maximum  of  90  milliseconds  for  the  hard  disk 

Reliability/ 

MTBF  2000  hours  minimum 

Maintainability: 

MTTR  0.5  hour  maximum 

Monitor. 

12  to  14  inch  (diagonal) 

Color 

EGA  Compatible  (non-interlaced) 

Minimum  flicker  (medium  to  long  persistence  phosphor) 

Medium  resolution  (640  x  480) 

9300  degree  Kelvin  color  temperature 

Low  glare  screen 

Meet  FCC  Part  15.  Subpart  J.  Class  A  emissions  (minimum) 

MTBF  of  15,000  hours  and  MTTR  of  0.5  hour 

Monitor  (Video): 

NTSC  compatible 

6500  degree  Kelvin  color  temperature 

All  other  specs  of  the  monitor  above 

Table  11-6  The  AOTS  Terminal  Specifications 


The  Computer  Assisted  Training  Subsystem  has  a  requirement  for  Interactive  Video 
Development  (IVD).  The  Sony  LDP2000/2  or  /3  were  considered  as  meeting  the  AOTS 
I V  D  needs.  This  is  the  same  machine  that  is  implemented  into  the  ISS  software.  The 
Eletrohomc  color  monitor  with  touch  screen  will  also  be  used.  Touch  screen  will  not  be 
used  in  the  prototype  AOTS.  The  color  graphics  adapter  used  is  manufactured  by  Matrox. 
This  is  the  same  adapter  as  used  by  the  U.S.  Army’s  EIDS. 
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COMPONENT 

REQUIREMENTS 

Optical  Mark  Sense 
Reader 

Optical  technology  for  reading  forms 

12  inches  per  second  scanning  rate  minimum 

8.5  x  11  inch  paper  size 

Bubble  matrix  80  x  25  x  2  minimum  sensors 

RS232C  interface  (two  or  pass  through) 

Meet  FCC  Part  15,  subpart  J,  Class  A  (minimum) 

Printer,  Color 

Minimum  of  4  colors 

Support  graphics  (screen  dumps  from  EGA) 

BlaaectionaJ  printing 

Multi-densities 

Dot  matrix  100  dots  par  inch  minimum 

Printing  spaed,  high  -  100  characters  par  second  (CPS),  low  -  35  CPS  | 

Regular  bond  paper  (20#)  compatible  | 

ParaBei  (Centronics  compatible)  and  RS232  interlaces 

MTBF  2000  hours,  MTTR  0.5  | 

Meet  FCC  Part  15,  subpart  J,  Class  A  (minimum)  f 

Printer  Monochrome 
-  Laser 

Dot  matrix  using  a  laser  f 

Text  and  graphics  capable 

300  dots  to  the  inch  minimum 

8  pages  per  minimum  speed 

Parade!  (Centronics  compatible)  and  RS232  interlaces 

MTBF  2000  hours,  MTTR  0.5 

Hold  a  minimum  of  500  sheets  of  8.5  x  11  bond  (#20)  paper  (plain  copier  type) 
Support  up  to  five  different  fonts  at  one  time 

Noise  level  not  to  exceed  55  db 

Meet  FCC  Part  15,  subpart  J,  Class  A  (minimum) 

Printer  Monochrome 
•  impact  dot  matrix 

Dot  matrix  using  pin  head  impact  technology 

Text  and  graphics  capable 

Parallel  (Centronics  compatible)  and  RS232  interfaces 

MTBF  2000  hours,  MTTR  0.5 

Have  tractor  feed  capability  for  conventional  computer  paper 

Not  exceed  70  db  noise  level 

Meet  FCC  Part  15.  subpart  J,  Class  A  (minimum) 

Digitizer  Tablet: 

11  x  11  inch  digitizing  area  (type  1) 

20  x  20  inch  digitizing  area  (type  2) 

RS232  interface 

Resolution  of  .005  inch 

Accuracy  of  .005  inch 

Cursor  puck  with  a  minimum  of  2  control  buttons 

Meet  FCC  Part  15,  subpart  J.  Class  A  (minimum) 

General  Specifications 
for  all  items 

All  items  must  be  sate  to  operate  and  handle 

Easy  to  transport 

Meet  office  environment 

Operating  temperature  range: 

65  to  r5  degrees  Fahrenheit  for  the  computer 
system 

55  to  90  degrees  Fahrenheit  for  all  other  items 

20  to  80  percent  relative  humidity  (non  condensing) 

Table  11*7  AOTS  Peripheral  Specifications 


The  Zenith  contract  also  eliminated  the  need  for  too  extensive  trade  studies  for 
printer  selection  (see  Table  11-7  for  key  specification  requirements  for  printers).  The 
Alps  P2000G  dot  matrix  printer  was  determined  to  meet  most  of  the  printer  needs  for 
AOTS.  For  development  purposes  (both  software  and  data  base),  high  speed  printing  was 
needed.  Requirements  for  development  of  documentation  along  with  the  high  speed 
printing  need  led  to  the  selection  of  laser  printers.  The  Hewlett  Packard  LaserJet  +  500 
printers  and  Laserjet  II  have  been  used  for  these  latter  requirements.  A  limited  number 
of  Okidata  Okimate  20  color  printers  were  procured  for  getting  color  screen  dumps  from 
the  Zenith  Z-248s. 
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Trade  studies  were  also  conducted  for  selecting  optical  mark  sense  readers  (OMR) 
and  digitizer  tablets  (see  Table  11-7  for  key  specification  requirements).  The  OMR 
selection  is  limited  by  the  small  number  of  possible  vendors.  The  process  selected  the 
Scantron  Corporation,  model  5200  OMR.  Two  types  of  digitizer  tablets  were  selected. 
One  was  a  20  inch  by  20  inch  tablet  for  large  drawings  being  digitized  and  the  other  was 
an  1 1  by  1 1  inch  tablet  that  was  also  part  of  the  standard  Zenith  contract. 


11.5.2  Selection  Process 

Criteria  for  selection  varied  somewhat  with  each  type  of  equipment.  Key  factors  in 
the  process  were: 


a)  Meeting  AOTS  technical  requirements 

b)  Cost 

c)  Reliability,  maintainability,  support  (logistics) 


One  of  the  major  influencing  factors  was  the  Air  Force  standard  contract  with 
Zenith  and  existing  Air  Force  assets  (e.g.  the  DEC  VAX  8600  computer).  Elsewhere,  the 
lowest  cost  approach  meeting  technical  requirements  guided  procurements.  Douglas 
Aircraft  Company  procured  the  communications  (leased  the  lines  from  Clay  Desta  and 
purchased  the  modems  and  multiplexers  from  Infotron),  the  optical  mark  sense  readers 
(Scantron),  laser  printers  (Hewlett  Packard),  color  printers  (Okidata),  and  the  large 
digitizer  tablets  (Summagraphics).  All  procured  items  will  become  Air  Force  inventory  at 
the  conclusion  of  the  contract  with  Douglas  Aircraft  Company. 


1 1 .5.3  Phased  Procurements 

Hardware  was  procured  in  two  basic  increments.  The  first  increment  of  hardware 
procured  was  to  meet  the  development  phase  needs  (Phase  II).  The  second  procurement 
was  to  meet  the  remaining  needs  for  SLT&E  (Phase  III). 


11.5.4  Reliability,  Availability,  and  Maintainability 


A  tailored  reliability  program  in  accordance  with  MIL-STD-785  will  evaluate  system 
reliability,  identify  problems,  and  recommend  solutions.  At  this  time  all  data  are  stated 
as  goals,  as  opposed  to  hard  requirements,  due  to  the  developmental  nature  of  the 
prototype  AOTS.  The  results  from  the  prototype  will  be  used  in  specifying  the  production 
version  that  will  be  deployed  throughout  the  Air  Force. 

The  data  will  be  judged  against  a  minimum  reliability  of  0.8  for  an  8  hour 
workcenter  shift,  7  days  per  week,  52  weeks  (a  year)  at  a  90%  confidence  level.  The 
following  techniques  will  be  used  to  achieve  required  levels  of  reliability. 

a)  Use  of  various  diagnostic  routines 

b)  Use  of  automatic  error  checking  (built  into  hardware  where  applicable) 

c)  Modular  design  (board  level  repair/replacc) 

d)  Component  failure  messages  to  operators  (via  reports,  logs,  direct  displayed 
messages,  etc.) 
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e)  Identification  of  impending  failures  (i.e.,  graceful  degradation) 

Mission  reliability  prediction  will  be  performed  in  accordance  with  MIL-STD-785B, 
paragraph  50.2.2.3. 


11.5.4.1  Reliability  Data 

The  overall  design  of  the  AOTS  ensures  system  reliability.  System  reliability,  aside 
from  hardware  and  software  concerns,  will  be  tested  at  the  user  interface  level  during 
SLT&E.  System  reliability  issues  center  around  the  concern  users  have  for  the  consistency 
and  accuracy  of  output  after  input  has  been  processed.  The  users  are  also  concerned  as  to 
how  easily  system  corrections  can  be  made  if  processing  problems  are  discovered.  Users 
of  AOTS  will  review  trainee  and  system  status  data  to  determine  if  the  system,  as 
designed,  is  working  consistently  and  accurately  (i.e.,  reliably).  For  example,  if  trainees 
are  failing  Quality  Control  evaluations  at  unacceptable  rates,  then  it  is  apparent  that 
AOTS  is  not  functioning  properly  (i.e.  it  is  not  consistent:  trainees  are  being  certified  but 
cannot  perform  their  tasks)  a s  a  system.  This  information  triggers  the  problem-solving 
process.  Maintainability  allows  the  user  to  trace  effects  to  causes  for  this  phenomenon. 
Table  11-8  provides  the  prototype  AOTS  goals  for  reliability. 


|  Parameter 

Goal 

MTBF 

250  Hours 

MTBDE 

400  Hours 

Mission  Reliability 

99.5% 

MCT 

0.5  Hours 

M  max 

0.5  Hours 

Mptmax 

32  Hours 

Ai 

99.5% 

Aa 

99.4% 

Ao 

99.8% 

Service  Life 

10  Years 

Operating  Life 

Operating 

25.000  Hours 

On/Oft  Cycles 

5.000  Cycles 

Time  Base  (for  calculations) 

24  Hours 

Table  11-8  Reliability  Goals  for  the  Prototype  AOTS 


1 1 .5.4.2  Maintainability  Data 

Table  11-9  lists  the  goals  for  the  maintainability  of  the  AOTS  hardware  equipment. 
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Parameter 

Goal  | 

MTTR  on  the  System 

1.5  Labor  hours 

90%  upper  limit  to  repair 

0.5  Labor  hours 

Mean-Time-To-Restore  the  system 

0.5  Labor  hours 

Mean-Time-To-flestore  the  system  -  90%  upper  limit  to 
restore 

1.0  Labor  hours 

Mean-Time-To-Remove  and  Replace 

0.4  Labor  hours 

Mean -Tim  e-To-flemove  and  Replace  -  90%  upper  limit  to 
Remove  and  Replace 

1.25  Labor  hours 

Direct  maintenance  manhours  per  equipment  operating 
hour 

0.05  Hours 

Response  time  to  site  of  failure 

4.0  Hours 

Principal  Period  of  Maintenance 

16  Hours/day 

7  days  per  week 

01  line  maintenance  average  MTTR 

6.0  Labor  hours  | 

9  %  upper  limit  to  repair 

6.0  Labor  hours  | 

f  ean  preventive  maintenance  time 

3  Hours  jj 

F  jan  Maintenance  Time 

3.4  Hours  | 

Logistics  Delay  Time 

4  hours  (90%tile)  | 

Maximum  corrective  maintenance  time 

8  Hours  | 

Administrative  delay  time 

2  Hours 

Maintenance  downtime 

7.4  Hours 

Maintenance  manhours/month 

6  Hours/Month 

Frequency  of  Preventive  Maintenance 

1  time  per  month  per  workcenter 

Table  11-9  Maintainability  Data  on  the  Prototype  AOTS 


The  following  factors  have  been  used  in  the  prototype  AOTS  using  MIL-STD-470, 
MlL-STD-472,  MIL-STD-421,  and  MIL-STD-1472  as  guidelines: 

a)  Use  of  built-in  tests  for  rapid  detection,  isolation  and  correction  of  failures. 

b)  Use  of  functional  reverification  to  ensure  proper  operation. 

c)  Use  of  standard  tools  and  equipment  to  maximize  interchangeability. 

d)  Exchange  of  failed  items. 

c)  Use  of  modular  design  to  enhance  diagnostic  capabilities. 

AOTS  has  been  designed  and  developed  to  ensure  system  maintainability.  System 
maintainability,  with  the  exception  of  software  and  hardware,  will  be  tested  at  the  user 
interface  during  the  development  and  implementation  of  the  prototype  AOTS.  A  key 
aspect  of  maintainability  is  traceability.  For  hardware  and  software,  documentation 
allows  one  to  trace  from  effects  back  to  causes.  Similarly,  AOTS  products  will  be 
associated  with  one  another  by  referential,  or  linking,  identifiers.  For  example,  task 
information  will  be  linked  to  behavioral  objectives,  b  avioral  objectives  to  the  test  items, 
test  items  to  evaluation  instruments,  etc.  These  connections  provide  a  context  for  any  one 
element  and  can  help  those  responsible  for  system  maintainability  to  adapt  the  system  to 
changing  conditions  or  to  isolate  deficiencies  that  result  in  unreliable  outputs.  As  system 
deficiencies  are  detected  through  the  use  of  system  reports,  problem  areas  can  be 
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identified  and  solutions  recommended.  These  solutions  will  be  tested,  new  data  collected, 
and  system  reports  generated.  This  cycle  of  monitor  for  reliable  outputs  (i.e.,  review 
reports),  document  changes,  and  monitor  again  is  the  application  of  maintainability,  or 
maintenance.  Following  the  installation  of  the  prototype  AOTS,  systems  reports  will 
provide  insight  into  how  maintainable  the  AOTS  really  is. 


1 1 .5.4.3  Availability  Data 

The  prototype  availability  information  is  derived  from  the  reliability  and 
maintainability  data.  This  data  is  taken  from  actual  maintenance  actions  on  AOTS 
equipment.  The  goals  for  the  availability  of  AOTS  hardware  are  provided  in  Table  11-10. 


Parameter  Goal 


System  wide  availability  I  96.5% 


Computer  System  Component  availability  98% 


Terminal  Component  availability  98% 


Printer  Component  availability  I  98% 


Digitizer  Pad  Component  availability  I  98% 


Optical  Mark  Reader  availability  96% 


Remote  Communications  95% 


Table  11-10  The  Prototype  AOTS  Availability  Goals 


11.5.4.4  Software  Maintainability 

Both  weekly  and  daily  backups  of  the  AOTS  software  will  be  performed  by  the 
AFHRL  Technical  Support  (TS)  personnel.  AFHRL-TS  will  also  implement  a  disaster  plan 
(i.e.  have  backup  copies  stored  either  in  a  secure  area  or  at  a  different  building  from 
where  the  computer  system  is  located). 

Software  maintenance  also  includes: 

a)  Updates  and  new  releases  of  system  software  (including  support  software 
packages  -  e.g.  Ada  compiler) 

b)  Restoration  of  all  data  should  a  failure  in  the  disk  subsystem  occur  or  a 
system  "crash"  result  in  loss  of  all  data  on  a  disk  subsystem. 


11.5.4.5  Performance  Goals 

Various  performance  (responsiveness)  goals  have  been  set  for  the  prototype  AOTS 
system.  With  the  system  operating  under  normal  conditions,  the  average  response 
performance  goals  for  a  user’s  terminal  would  be  2.5  seconds. 
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11.6  Software  Architecture 


11.6.1  Software  Development 

The  prototype  AOTS  is  using  the  MIL-STD-1815A  Ada  computer  programming 
language.  AOTS  represents  a  major  Air  Force  program  using  Ada  with  over  400,000  lines 
of  source  code.  Approximately  half  of  this  software  was  newly  developed  for  the 
prototype  AOTS.  The  approximate  other  half  came  from  the  AFHRL  Instructional 
Support  System  (ISS)  project.  ISS  is  also  an  AFHRL  major  Ada  software  project.  In 
addition  to  Ada  language,  an  Ada  Design  Language  (ADL)  is  also  being  used. 

The  prototype  AOTS  software  is  broken  down  into  Computer  Program  Configuration 
Items  (CPCIs'.  The  AOTS  system  consists  of  the  following  CPCIs: 


a)  Management  (Subsystem)  CPCI 

b)  Evaluation  (Subsystem)  CPCI 

c)  Training  Development  and  Delivery  (Subsystem)  CPCI  (which  is  ISS  software 
intact)  NOTE:  This  is  referred  to  as  the  Computer  Assisted  Training 
Subsystem  within  the  body  of  the  Operational  Guide. 

d)  System  Support  CPCI  (part  comes  from  ISS  and  the  rest  is  newly  developed 
software  for  the  prototype  AOTS) 


11.6.2  Software  Architecture 


The  prototype  AOTS  software  architecture  is  based  on  a  central  computer  system 
architecture.  Figure  11-5  provides  a  "layered"  diagram  approach  to  how  the  AOTS 
software  is  engineered.  The  central  core  of  the  system  is  the  DEC  VAX  8600’s  computer 
operating  system,  VMS.  The  VAX  8600  VMS  operating  system  is  a  virtual  memory 
multiuser  and  multitasking  operating  system.  Immediately  outside  the  operating  system  is 
the  software  that  interfaces  other  AOTS  software  with  the  operating  system  itself.  This 
software,  the  system  support  layer,  interconnects  with  the  subsystems  soltware  through  an 
intermediate  layer.  Figure  11-6  provides  a  vertical  approach  to  diagraming  the 
relationships.  The  outside  layer  is  referred  to  as  the  application  layer.  This  is  where 
most  of  the  "editors"  are  located  as  well  as  other  user  interfaces. 

The  Zenith  Z-248  Personal  Computer  (PC)  also  is  programmed  in  Ada.  The  Alsys 
Corporation’s  validated  compiler  was  utilized.  The  operating  system  utilized  on  the 
prototype  AOTS  for  the  Z-248  is  Microsoft’s  MS-DOS  version  3.1.  The  Z-248  software 
developed  for  AOTS  provides  a  "terminal  emulation"  capability.  The  terminal  emulator 
takes  data  received  from  the  VAX  and  interprets  and  displays  it  on  the  Zenith  Z-248 
hardware.  A  user  can  input  data  through  the  keyboard  back  to  the  VAX  computer. 


Figure  11-6  A  Sample  Vertical  Interface  Diagram 

The  Terminal  Editor  is  used  to  denote  a  software  package  that  allows  a  user  to  enter 
or  alter  data. 

Various  examples  of  the  editors  have  been  given  in  the  main  body  of  the 
Operational  Guide  document. 
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11.6.3  Standards  Used 

The  prototype  AOTS  is  using  the  MIL-STD-1815A  Ada  specifications.  In  addition, 
AOTS  is  using  programming  standards  developed  specifically  for  this  program.  These 
AOTS  programming  standards  expand  upon  the  MIL-STD  by  adding  the  Ada  Design 
Language  (ADL),  AOTS  standards  package  header  documentation,  and  AOTS  standard 
procedure/function  header  documentation. 


AOTS  is  also  using  various  limits  that  help  in  defining  the  AOTS  software.  A  major 
example  of  these  are  the  limits  placed  on  a  Ada  package  and  module.  An  Ada  package  is 
limited  (with  exceptions)  to  being  1  to  2  thousand  lines  of  code.  An  AOTS 
procedure/function  is  limited  (with  some  exceptions)  to  100  lines  of  code. 


1 1 .6.4  Computer  Program  Structure 

The  prototype  AOTS  computer  programs  consist  of  four  Computer  Program 
Configuration  Items  (CPCIs),  and  various  Ada  packages  and  Ada  procedures/functions. 


A  Computer  Program  Configuration  Item  (CPCI)  is  the  actual  computer  program  end 
item  in  the  form  of  computer  instructions  stored  on  machine-readable  media.  A  CPCI 
will  consist  of  one  or  more  Ada  packages. 

An  Ada  procedure/function  performs  a  complete  logical  process  by  execution  of  a 
set  of  instructions  which  have  clearly  defined  inputs,  processing  logic,  and  outputs.  A 
procedure/ function  is  the  smallest  set  of  executable  statements  able  to  be  compiled.  Each 
module  will  conform  to  the  following  conventions: 

a)  A  proccdure/function  consists  of  a  set  of  instructions  in  a  form  consistent 
with  the  Ada  language,  VAX/VMS  operating  system,  and  the  VAX  8650 
computer  system. 

b)  A  procedure/function,  with  some  exceptions,  does  not  exceed  100  lines  of 
executable  source  code.  This  limitation  excludes  comments  and  data  defini¬ 
tions. 

c)  A  procedure/function  has  only  one  entry  statement  and  one  exit  statement. 


11.6.5  Top  Down  Design  (TDD) 

The  overall  approach  to  designing  the  AOTS  software  has  been  to  use  Top  Down 
Design  (TDD)  methodology.  The  processing  activities  of  the  system  are  identified  and 
organized  beginning  with  higher  levels  of  organization,  (i.e.,  top  levels).  These  higher 
levels  are  then  expanded  and  broken  out  to  include  a  more  detailed  definition  of  the 
processing  activities  by  identification  of  subordinate  levels.  The  lowest  level  of 
processing  corresponds  to  the  procedure/function. 


11.6.6  Top  Down  Implementation  (TDI) 

The  AOTS  software  has  been  implemented  in  a  top  down  manner.  Conceptually,  top 
down  implementation  proceeds  from  a  single  starting  point  while  conventional 
implementations  proceed  from  as  many  starting  points  as  programs  in  the  design.  The 
single  starting  point  does  not  imply  that  the  implementation  has  proceeded  down  the 
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hierarchy  in  parallel.  For  example,  user  or  other  external  interfaces  might  be 
implemented  before  some  of  the  other  portions  to  permit  early  usage  of  the  software 
subsystem  capabilities,  partial  software  system  evaluation,  and  training. 


11.6.7  Structured  Coding  and  Operating  System  Requirements 

Only  well  defined  control  constructs  have  been  used  in  deve^ping  the  AOTS 
software.  These  constructs  use  logically  equivalent  Ada  language  simulations.  Some 
examples  of  these  constructs  are:  IF-THEN-ELSE,  DO,  and  CASE. 

The  operating  systems  used  (on  the  DEC  VAX  8600,  VAX/VMS;  and  on  the  Zenith 
PC,  MS-DOS)  are  existing  off-the-shelf  software  packages.  Changes/augmentations  will  be 
permitted  to  be  embedded  within  vendor  supplied  operating  system  (OS)  software.  No 
AOTS  changes/augmentations  will  affect  the  vendor  supplied  software,  thus  the 
maintenance  of  this  software  will  be  provided  by  the  vendor  for  the  life  cycle  of  the 
system. 

The  program  coding  conventions  that  have  ttfen  used  include: 

a)  Each  line  of  source  code  contains  no  more  than  one  statement. 

b)  Source  code  :s  clearly  and  conspicuously  annotated  to  explain  all  inputs, 
outputs,  branches,  and  other  items  not  implicit  in  the  code  itself. 

c)  Names  of  operator  commands,  data  entries,  program  components,  variables, 
procedures,  and  other  software  components  are  consistent  with  those  used 
elsewhere  in  the  systems  design. 

d)  Code  has  been  written  such  that  no  code  is  modified  during  execution. 


11.6.8  Unit  Development  Folders 

As  part  of  the  development  effort.  Unit  Development  Folders  (UDFs)  were 
maintained  for  each  Ada  package.  The  UDFs  contain  the  design,  source  code  listings,  test 
procedures,  test  results,  and  history  on  each  package.  The  UDFs  arc  used  tor 
configuration  control  on  each  Ada  package. 


11.6.9  Utility  Software  Services 

The  support  utility  software  provided  includes  such  capabilities  as: 

a)  Validated  Ada  compilation  capabilities 

b)  Assembly  which  produces  relocatable  object  code 

c)  Linking  type  loader 

d)  Generation,  maintenance,  and  initialization  of  storage  media  for  programs  and 
data 

c)  Diagnostics  to  support  fault  isolation 
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f)  Editing  and  debugging  tools 

g)  Source  code  configuration  control  tools 

h)  Error  handling  (error  detection  and  reporting)  and  logging  of  computer 
internal  problems 

11.6.10  Usage  of  ISS 

Approximately  half  of  the  AOTS  software  comes  from  the  ISS  program.  All  of  the 
Computer  Assisted  Instruction  (CAI)  and  some  of  the  system  support  software  is  taken 
from  ISS  intact. 
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11.7  Glossary  Of  Terms  And  Acronyms  -  APPENDIX  B 


ACRONYM 

DEFINITION 

ADL 

Ada  Design  Language 

AF 

Air  Force 

AFB 

Air  Force  Base 

AFHRL 

Air  Force  Human  Resources  Laboratory 

AFHRL/IST 

Air  Force  Human  Resources  Laboratory/Instructional 
Support  Team 

AFHRL/OL-AK 

Air  Force  Human  Resources  Laboratory/Operating 
Location  -  AK  (Berg  from  AFB,  Texas) 

AFHRL/TS 

Air  Force  Human  Resources  Laboratory/Technical 
Services  Division 

AFR 

Air  Force  Reguiatio 

Aa 

Availability,  achieved 

Ai 

Availability,  inherent 

Ao 

Availability,  operational 

ANG 

Air  National  Guard 

ANGB 

Air  National  Guard  Base 

AOTS 

Advanced  On-the-job  Training  System 

ATR 

Airman  Training  Record 

BLDG 

Building 

BO 

Behavioral  Objective 

CAI 

Computer  Assisted  Instruction 

CGA 

Color  Graphics  Adapter  -  a  standard  of  resolution  on 
IBM  PCs 

CPC 

Computer  Program  Component 

CPCI 

Computer  Program  Configuration  Item 

CPS 

Characters  Per  Second  (when  used  with  printers) 

DAC 

Douglas  Aircraft  Company 

db 

Decibels 

DEC 

Digital  Equipment  Corporation 

DG 

Data  General  Corporation 

EA 

Each 

e.g. 

example 

EGA 

Extended  Graphics  Adapter  -  a  standard  of  resolution 
on  IBM  PCs 

EIDS 

Electronic  Instructional  Development  System 

F 

Fahrenheit 

FCC 

Federal  Communications  Commission 
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Ft 

Feet 

FIG 

Figure 

IBM 

International  Business  Machines  Corporation 

i.e. 

that  is 

IM 

Inventory  Manager 

I/O 

Input/Output 

ISS 

Instructional  Support  System 

ITR 

Individual  Training  Requirements 

IVD 

Interactive  Video  Development 

KB 

Kilobaud 

LAN 

Local  Area  Network 

M  max 

Maintenance  maximum 

MCT 

Maintenance,  complete  time 

MB 

Megabytes 

MHz 

MegaHertz 

MIL-STD 

Military  Standard 

MIPS 

Millions  of  Instructions  Per  Second 

Mptmax 

Maintenance  preventive  time  maximum 

MTBF 

Mean  Time  Between  Failures 

MTL 

Master  Task  List 

MTTR 

Mean  Time  To  Repair 

MUX 

Multiplexer 

NTSC 

National  Television  Standard  for  Color  transmission 

OJT 

On-the-Job  Training 

OMR 

Optical  Mark  Reader  or  Optical  Mark  Sense  Reader 

OS 

Operating  System 

PC 

Personal  Computer 

PCs 

Personal  Computers 

PGA 

Professional  Graphics  Adapter  -  a  standard  of  resolu¬ 
tion  on  IBM  PCs 

PRES 

Presentation 

QA 

Quality  Assurance 

QTY 

Quantity 

RES. 

Reserves 

SHM 

Short  Haul  Modem 

SLT&E 

System  Level  Test  and  Evaluation 

SU 

System  Unit 

TDD 

Top  Down  Design 

TDI 

Top  Down  Implementation 

TFG 

Tactical  Fighter  Group 

TX 

Texas 

UDFs 

Unit  Development  Folders 
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UDS 

Universal  Data  Systems 

U.S. 

United  Statues 

UTA 

Unit  Training  Activity 

VAC 

Volts,  Alternating  Current 

VAX 

Virtual  Address  Extension  •  A  product  designation  for 
a  line  of  computer  manufactured  by  Digital  Equipment 
Corporation 

VAX/VMS 

Virtual  Address  Extension/Virtual  Memory  System  - 
the  operating  system  designation  used  on  a  line  of 
computer  manufactured  by  Digital  Equipment  Corpora¬ 
tion 

%tile 

percentile 
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